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ABSTRACT 


In the Federal Acquisition Regulations (FAR) Statement of Guiding Principles for the 
Federal Acquisition System, the vision of the Federal Acquisition System is to deliver best 
value products or services to the customer. Contracting Officers must achieve this while 
balancing the many competing interests of the stakeholders in the System. The paradox of 
efficiency vs. effectiveness can be found in the second sentence by the phrase “balancing the 
many competing interests in the System”. This statement indicates the diverse interest of the 
many stakeholders involved in the System that in many instances prevent the Contracting 
Office from being efficient and effective. The Government Performance Results Act of 1993 
also requires each agency to establish projected outcomes or results by which they will be 
evaluated against. This thesis examines various literature and existing measurement systems 
of field contracting offices to determine if we are properly evaluating efficiency and 
effectiveness. The thesis also utilizes the Organizational Configuration Model developed by 
Nancy Roberts to determine where field offices fit. The thesis identifies common themes 
found in metrics and draws conclusions based on that information. Finally, the researcher 
proposes a model for Field Contracting Offices to use for evaluating their efficiency and 
effectiveness. It is the researcher’s hope that this thesis will be of benefit to all field 
contracting offices that struggle with determining their efficiency and effectiveness. In 


addition, it is hoped that Systems Commands find some useful information in this thesis. 
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I. INTRODUCTION 


A. BACKGROUND 

Why is efficiency and effectiveness important to a Contracting Office? In the 
Packard Commission’s (1986 p.42) final report to the President it states: 

We must give acquisition personnel more authority to do their jobs. We 

must make it possible for people to do the right thing the first time and 

allow them to use their common sense. When this is done, layers of 

supervision can be eliminated, reporting can be minimized, and the 

Department of Defense (DOD) can get by with far fewer people. Only 

then will productivity and quality become hallmarks of defense 

acquisition. 

This statement gives a clear indication that the System must become more efficient. 
This is further reinforced in the Federal Acquisition Regulations (FAR) Statement of Guiding 
Principles for the Federal Acquisition System, namely, the vision of the Federal Acquisition 
System states: 

All participants in the system are responsible for making acquisition 

decisions that deliver best value product or service to the customer. Best 

value must be viewed from a broad perspective and is achieved by 

balancing the many competing interests in the System. The result is a 

system which works better and costs less. (FAR 1.102-1) 

The paradox of efficiency vs. effectiveness can be found in the second sentence by 
the phrase “balancing the many competing interests in the System”. This statement indicates 
the diverse interest of the many stakeholders involved in the System that in many instances 
prevent the Contracting Office from being efficient and effective. In Chapter II this will be 
discussed in further detail when efficiency and effectiveness are defined. The guiding 
principles also tells us how to achieve efficient operations: 

To achieve efficient operations, the System must shift its focus from “risk 

avoidance” to one of “risk management.” The cost to the taxpayer of 

attempting to eliminate all risk is prohibitive. The Executive Branch will 

accept and manage the risk associated with empowering local procurement 


officials to take independent action based on their professional judgment. 
(FAR 1.102-2) 


In other words, empower the Procurement Contracting Officer (PCO), and then trust 
him/her to do the right thing. The guiding principles also empower the PCO “to innovate and 
use sound business judgment” when a policy or procedure is not specifically addressed in the 
FAR. In theory, this seems to give the PCO wide latitude to exercise his discretion in making 
business decisions. In reality, this premise is largely dependent on the support the PCO 
receives from his upper management and agency hierarchy. As will be discussed later in 
Chapter II this support has a major impact in a contracting office’s efficiency and 
effectiveness. Indeed, the Federal Government is enamored with the idea of an efficient and 
effective acquisition system. The decade of the 1990s led to some significant legislation to 
promote a more efficient and effective acquisition system. This legislation included: 

- Government Performance Results Act (GPRA) of 1993 

- Federal Acquisition Streamlining Act (FASA) of 1994 

- Federal Acquisition Reform Act (FARA) of 1995 

- Clinger-Cohen Act of 1996 

- FAR Part 15 Rewrite of 1997 
All of these legislations sought to make the acquisition system more efficient and effective 
by focusing on commercial practices. However, the Federal Government cannot completely 
adopt all of the practices of the commercial sector. The Procurement Round Table (PRT) 
echoed this point in their 2001 report: 

The Federal Government has been and always will be different from the 

commercial sector. Thus, while striving to make the Federal Acquisition 

System more like its commercial counterpart, the PRT continues to 

recognize the unique constraints that are imposed on a public sector 

organization. No matter how commercial, competitive, or cost effective 

the Federal Acquisition Systems becomes, it ultimately will still be 

governed by public policies—policies that are driven not only by 

economic objectives, but also by social and political considerations. 

Despite these limitations, the Federal acquisition system must remain 

properly focused on mission accomplishment. (The Federal Acquisition 

System: Transitioning to the 21“ Century (p.6)) 

As shown above, as a public entity, becoming efficient and effective is important to 
its ultimate stakeholder, the taxpayer. 

The problem of measuring efficiency and effectiveness of any organization seems to 


span the entire public sector. The Government Performance Results Act (GPRA) of 1993 put 
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forth a requirement that organizations begin to measure themselves against desired outcomes. 
In other words, their performance and existence will be measured on their results. 
Implementation of the Results Act has faced a number of challenges since enactment. These 
challenges have been well documented in various GAO reports (GAO/T-GGD-96- 136; 
GAO/T-GGD-97-83; GAO/T-GGD-98-44; GAO/T-GGD-98-66). GAO auditors 
acknowledge that it will take time to orient federal managers to a new way of planning and 
managing and develop routines that enable them to process information and make informed 
decisions about what they do and how they do it. (GAO/T-GGD-97-113). Most Defense 
activities still have difficulty linking subjective mission goals to objective outcomes. 

Now that we have looked at why efficiency and effectiveness is important, Chapter II 
will focus on what we mean by efficiency and effectiveness. 

The purpose of this thesis is to analyze the metrics a Field Contracting Office utilizes 
to determine their effectiveness and efficiency. In addition, a metrics model will be 
developed and recommended for implementation 

The basic goal of this thesis is to propose a model that a contract office may use in 
determining their efficiency and effectiveness. Included in that will be an analysis of 
relevant metrics that help to guide organizations to achieve them. 

B. RESEARCH QUESTIONS 

1. Primary Research Question 

What model is needed to measure the efficiency and effectiveness of a Navy Field 

Contracting office? 

2. Secondary Research Questions 

a. What is the difference between efficient and effective and why should we 


strive for both? 


b. Where do most Field Contracting Offices fit in the Organizational 
Configuration model? 


c. What metrics are typically used to measure Field Contracting efficiency and 
effectiveness and what are the issues and/or limitations associated with these 
metrics? 

d. What type of metrics model is required in order to measure the efficiency and 


effectiveness of a Navy Field Contracting Office? 


C, METHODOLOGY 

The proposed area of research for this thesis concerns the efficiency and effectiveness 
of a Navy Contracting Office. Specifically, the research will attempt to define the concepts 
of efficiency and effectiveness as well as propose a set of metrics or a model that may be 
used in measuring the effectiveness and efficiency of a Navy Field Contracting Office. 

This study will require an extensive review of existing literature as well as previous 
studies performed on this topic. In addition, it will research the Organizational Configuration 
model as depicted by Professor Nancy Roberts and analyze where Contracting Offices fit on 
the Organizational Model and recommend ways to optimize efficiency and effectiveness. 
Various metrics will be studied to determine their effectiveness in measuring the success of a 
contracting organization. Also, the research will look at the benefits of the balanced 
scorecard approach in measuring organizational success. The researcher is in the unique 
position of being responsible for establishing a metrics program and quality assurance 
program at the Fleet Industrial Supply Center (FISC) Norfolk Detachment Philadelphia, and 
as such has extensive access and insight into the current process, metrics, and other data. 

The theoretical framework in which the study will be performed will be centered 
around first defining the organizational configuration model of the contracting office as 
depicted in Nancy Robert’s research on Organizational Configuration. By first doing this, 
we can examine the tradeoffs between efficiency and effectiveness that a contracting office 
should consider. The researcher will then examine the balanced scorecard approach in 
measuring organizational success. Finally, the researcher will propose a model for 
measuring the effectiveness and efficiency of a contracting office. The researcher plans on 
using FISC Norfolk Detachment Philadelphia, as a sample of many of the contracting issues 
that effect a contracting organization. Additionally, the researcher will gather information 
from other contracting offices as well as look at several private sector organizations to 
determine a benchmark or possible best practices in organizing and measuring a contract 
office. 

The researcher will use a variety of methods to gather data. First a comprehensive 
review of the literature will be conducted that will include Department of the Navy, 
Department of Defense, other Federal Agencies, and commercial sources. This data will be 


used to provide background on the current state of measuring organizational performance. 
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The researcher will also analyze a summary of previous Navy Studies conducted over the last 
10 years. These studies will serve to identify problem areas and successes. Policies, best 
practices, General Accounting Office (GAO) reports, National Association of Purchase 
Managers (NAPM) research studies, the balanced scorecard approach, professional journal 
articles and previous theses will be reviewed, incorporated and cited as applicable. 
Electronic searches via the Internet will be a primary method of searching the literature. 

The researcher is in a unique position at FISC Norfolk Detachment Philadelphia that 
allows access to metrics data gathered at all levels of the organization. The researcher is 
currently responsible for implementing an overall quality assurance program at FISC Norfolk 
that is used by all of its detachments (Philadelphia, Washington, Hampton Roads, and 
Norfolk Naval Shipyard). In addition, as the Quality Assurance Program manager, the 
researcher is required to develop a metrics program that adds value and meets the 
requirements of FISC Norfolk as well as Headquarters, Naval Supply Systems Command 
(NAVSUP). The researcher plans on using resources at NAVSUP to help define the 
methodology they utilize in determining relevant metrics. The researcher also plans on 
contacting other Navy activities to understand their methodology in selecting metrics. This 
data along with informal interviews and personal observation will allow the researcher to 


provide a detailed description of the current environment at FISC Norfolk. 


The goal of this study is to provide a variety of benefits to FISC Norfolk and the 
Naval Supply Systems Command (NAVSUP) and maybe the Department of the Navy as 
a whole. First, the study should provide FISC Norfolk with a framework to implement a 
relevant metrics program. This should also provide headquarters with a framework and 
an understanding of the how their subordinate commands measure performance. This 
will hopefully provide NAVSUP with a standardized approach that they can implement 
to the entire field of contracting offices. This may also lead to NAVSUP recommending 
to the Assistant Secretary of the Navy (ASN) to implement this approach Navy wide. 


The researcher does not plan on using a survey. Many of the studies analyzed are 
recent and provide many anecdotal summaries. Given the researcher’s experience in this 
area, informal interviews with other contracting offices and benchmarking some industry 


practices will provide enough information to derive conclusions. 


Lastly, the study may identify current pathologies in the performance process that 
can lead to additional research in this area to gain additional efficiencies and process 
improvement 


D. SCOPE/LIMITATIONS/ASSUMPTIONS 


The general thrust of the study is to identify the relevant metrics you should 
examine to determine the performance of a Contracting Office. Problems of capturing 
non-value added measures are commonplace in today’s contracting organizations. 
Linking metrics to strategic mission and operations is disjointed. Also, establishing a tier 
metric approach is sometimes difficult to implement. What Headquarters needs is 
different than what the Commanding Officer (CO) needs, which is different than what the 
first line supervisor needs. A linking mechanism must be established so all proponents 
understand where they fit in the overall metric. In addition, the goal of all metrics is that 
they are value added and motivate positive performance. These issues are critical to 


understand when you are discussing the performance of a contracting office. 


This thesis will focus on measuring efficiency and effectiveness of a Navy Field 
Contracting Office. As such, major system acquisition and headquarters contracting 
offices are not included in this study. Since the complexity of work is so divergent in 
these types of Contracting Offices, it is difficult to establish a specific model for a 
headquarters command, since their metrics would be significantly different than a field 


contracting office. 


Another limitation on this thesis is that the researcher is recommending a specific 
model for a field contracting office. This will include some strategic metrics, but only as 
they relate to the Navy Field Contracting Office and their relationship with NAVSUP. 
The researcher will not attempt to implement specific strategic metrics for the Navy 
overall. The metrics limitations will be to a Navy Field Contracting Office. 


E. OVERVIEW AND ORGANIZATION 


This thesis is divided into four chapters. This chapter has provided background 
concerning the growing interest in contracting efficiency and effectiveness. In addition, 
the primary research question and secondary questions along with the scope, limitations 


and methodology have been discussed. 


Chapter II defines efficiency and effectiveness and utilizes the Organizational 
Configuration model as depicted by Professor Nancy Roberts. It analyze where 
Contracting Offices fit on the Organizational Model and recommend ways to optimize 
efficiency and effectiveness. Various metrics will be studied to determine their 
effectiveness in measuring the success of a contracting organization. Additionally, this 
chapter will also include an extensive review of existing literature, including an analysis 


of the balanced scorecard approach. 


Chapter HI presents data gathered via informal telephone interviews and a 
contractor’s (Booz-Allen Hamilton) summary of previous studies performed by or for the 
Navy. In addition, the researcher will examine a study performed by the Center for 
Advanced Purchasing Studies (CAPS) concerning effectiveness of a purchasing office. 
The researcher will then analyze the results of the data. This chapter will also provide the 
current metrics that FISC Norfolk and NAVSUP utilizes in measuring their field 
contracting offices and analyze the usefulness of these metrics as well as look at some 
current DCMA metrics and analyze their usefulness in a field contracting office. The 
researcher will also discuss the implications of the obtained results to the data collection 
process and analyze the results from the perspective of the literature review and the 


current state. 


Chapter IV provides conclusions based on the research results, a model for 
measuring the efficiency and effectiveness of a contracting office and identifies areas for 


further study. 


F. SUMMARY 

Efficiency and effectiveness is found almost everywhere when discussing 
Government operations. The question of how we achieve it and how we measure it continue 
to perplex most contracting offices. To achieve a proper balance between efficiency and 
effectiveness, a contracting office must know their stakeholders and must be able to balance 
their interests against the overall mission of their organization. Through the research efforts 
embodied in this thesis, a model will be proposed that field contracting offices can utilize in 


measuring their efficiency and effectiveness. 


Lastly, the study may identify current pathologies in the performance process that 
can lead to additional research in this area to gain additional efficiencies and process 


improvement. 


Il. CONTRACTING OFFICE EFFICIENCY AND 
EFFECTIVENESS BACKGROUND 


A. OVERVIEW OF A FIELD CONTRACTING OFFICE 


In order to define the mission of a Field Contracting Office, the mission of its 
headquarters command must be defined first. The Naval Supply Systems Command's 
(NAVSUP) primary mission is to support the Navy, Marine Corps, Joint and Allied 
Forces with high-quality supplies and services, whenever and wherever those supplies 
and services are needed. NAVSUP delivers combat capability through logistics. Navy 
NAVSUP receives its Contracting Authority from the Secretary of the Navy. NAVSUP 
is responsible for management of the Navy Field Contracting System (NFCS), which 
comprise NAVSUP's command activities as well as all other Navy field activities 
delegated contracting authority by COMNAVSUP, the Head of the Contracting Activity 
(HCA) for the NFCS. NAVSUP’s contracting directorate serves as the HCA principal 
staff for contracting policy matters, operational review and specific approval actions. 
Additionally, NAVSUP advocates and facilitates activities necessary to accomplish the 
cultural and process changes entailed in effecting the Acquisition Reform (AR) goals of 
reducing acquisition costs and process lead times while improving product and service 


availability, performance and reliability (http://www.navsup.navy.mil). Therefore, it is 





NAVSUP’s mission to facilitate the achievement of effectiveness and efficiency within 
their field offices. ASN further defines NAVSUP’s contracting responsibilities to include 
contracting for supplies and services throughout the Department of Navy for which no 
other contracting activity, office or command is delegated contracting authority. This 
includes contracting for Naval (but not Marine Corps) activities visual information 
(motion picture and videotape) productions through the Naval Media Center, unless an 
exception is granted by the Chief of Naval Operations (N09C4) (Naval Acquisition 
Procedures Supplement 5201.601(90)(c) (2)). 


A field contracting office is responsible for implementing NAVSUP’s contracting 
mission. A typical field contracting office supports both afloat and ashore activities and 
procures all types of supplies and services to support those customer commands. In 


addition, NAVSUP’s field offices obligate approximately 6 billion dollars a year on 
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Supplies and Services (Procurement Management Reporting System FY 02 Obligations). 
A NAVSUP field contracting office receives their contracting authority via an instruction 
(NAVSUPINST 4200.81 (series)). A field contracting office is responsible for 
negotiating, awarding, and administering contracts in addition to providing other related 
services to their customers on issues that relate to the myriad of regulations that govern 
the acquisition process. The goal of a field contracting office is to provide their customers 
with the best value products and services required to carry out their mission. As was 


stated earlier in the FAR’s guiding principles, the Contracting Officer must: 


a. Satisfy the customer in terms of cost, quality, and timeliness of the 


delivered product or service. 
b. Minimize administrative operating cost. 
C. Conduct business with integrity, fairness, and openness; and 
d. Fulfill public policy objectives. 


All of these requirements have an impact on the efficiency and effectiveness of a 
contracting office. There are constant tradeoffs that take place in order to satisfy the 


many stakeholders involved in the process. 


NAVSUP (as is the rest of the Navy) is currently undergoing a significant 
“Transformation” in business affairs to help reduce overall Navy-wide infrastructure 
costs by $10 Billion dollars. In the Jan/Feb 2003 Navy Supply Corps Newsletter, there 
appeared an article on “Transforming NAVSUP”. The article states, *NAVSUP’ has 
embarked on a journey to better structure and align the organization to most effectively 
and efficiently deliver combat capability through logistics”. In the Fleet Industrial 
Supply Center (FISC) Norfolk’s (the researchers parent command) presentation 
“Planning and Implementing NAVSUP Transformation”, they list the Transformation 


objectives: 
- Maintain ability to accomplish mission. 
- Seek opportunities to improve processes. 


- Gain efficiencies, which lead to reduced costs. 
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The goal in transformation is to become more efficient and effective by 
transforming the way we do business. There are several working groups analyzing all of 
NAVSUP’s functions to determine how to transform operations. Unfortunately, to the 
best of the researchers knowledge, productivity or efficiency of a contracting office is not 


being analyzed, only actions, commodities and geographical location. 


Now that the mission of a field contracting office has been defined, the question 
of what is efficiency and effectiveness needs to be answered. In Chapter III, NAVSUP’s 
current metrics for measuring the performance of their field contracting offices will be 
analyzed to determine if they are truly measuring the effectiveness and efficiency of their 
field contracting organizations. 


B. LITERATURE REVIEW ON EFFICIENCY AND EFFECTIVENESS IN 
PERFORMANCE MANAGEMENT 


The acquisition workforce has never had to account for its contribution to the 
overall defense strategy of the United States in an explicit, thoroughly rigorous manner. 
Since the end of the Cold War, however, the acquisition workforce has been pressured to 
justify itself both in terms of value creation and unique competency (Barzelay and 
Thompson, 2001). The real problem seems to be that the acquisition workforce has 
trouble explaining how they provide value. The importance of defining how a contracting 
office provides value to its customers cannot be understated. It is the crux for developing 
relevant and important metrics. This leads the contracting office to define what they do 
and why they do it. Value analysis requires you to seek ways to improve cycle efficiency 
(value-added time vs. total time) and determine ways to reduce associated costs due to 


delays, excesses, and non-value added processes (Barzelay and Thompson, 2001). 


The Federal Computer Week (May 2003) issued an article explaining the current 
administration’s desire for a performance-based system. According to Mark Froman, the 
administrator of the office of E-Government at the Office of Management and Budget, 
“The White House will continue to scrutinize agencies for measurable performance and 
increased efficiency”. The administration’s proposed 2004 budget ties agencies funding 
increasingly tightly to achievements. The 2004 budget specifically addresses five areas 


of continued weakness, he said: 
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Automating inefficient processes, or "paving cow paths." Simply 
converting a badly planned process from manual to automatic 
won't make the agency more effective. Agencies should rethink 
their processes and discover how to get the most from available 
technologies. 


Program management. Far too many programs still come in past 
deadline and over budget. 


Redundant buying. Agencies should find ways to consolidate 
operations so that they need fewer systems. 


Islands of automation. Agencies still haven't solved the age-old 
stovepipe problem. 


Cybersecurity. Keeping information protected while making it 
available to those authorized to have it is a never-ending concern. 


While the majority of his comments were direct to Information Technology (IT) 


investments, they also have relevance for non-IT investments. Most field contracting 


offices that were researched found the same problems in their organizations, especially 


the first and third bullets cited above. 


Graves (2001) defines Government efficiency as the ratio of outputs to inputs. It 


seems simple enough. However, the general lack of markets to assign value to public 


sector outputs make the process extremely difficult. Public sector pursuit of efficiency 


requires: 


Strategic planning to infer required outputs from politically 
determined outcomes and to make tradeoffs between the desired 
outputs to determine target levels of service; 


Accounting that allows quantifying the input cost that contributes 
to an output; 


Quantitative measure of output goals and performance; and 


Motivation and resources to establish the management systems and 
to make the choices that maximize efficiency. 


In Lloyd’s (Government Contracting Pathologies, 2000), he indicates that we 


should calculate whether some contracting offices have excess capacity. If so, transfer 


personnel to make more sensible distribution of workload. But how can a determination 


of excess capacity be done without an analysis of a contracting office’s efficiency. 
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In Chapter I we mentioned many of the acquisition reform legislation that was 
implemented in the 1990s. The main consideration in acquisition reform is whether the 
new process enables us to field better weapon systems, faster, and cheaper (Booher, 
Pinker, Smith 1997). This suggests that we must become more efficient in our operations 
and processes. In addition, in order to attain this objective and demonstrate that 
improvement and some efficiency has been achieved it is imperative that we establish 


some measurement for comparing the old process to the new. 


(Newcomer and Scheirer, January 2001) studied 23 federal agency’s 
implementation of the Government Performance Results Act (GPRA). In their report, 
they identified many different steps to performance-based management. They found that 
performance based management is much more than simply recording measures of 
program performance and reporting them upwards to oversight bodies and stakeholders. 
The following steps are needed to develop and collect performance measures useful for 
decision-making: 

- Program stakeholders must come together to reach agreement on 


strategic and performance objectives and the strategies for 
achieving them. 


- Indicators must be defined for program components that capture 
program outputs and/or outcomes. 


- Data sources must be developed or discovered for those indicators 
and data must be aggregated and reported in user-friendly formats. 


- Data must be used by decision makers to assess and improve 
results and the data must be addressed at every step in the process 
from original collection to final reporting. 


In addition, evaluators must have the skill set to accomplish all of these steps. 
This requires significant analytical capacity. Unfortunately, the Newcomer and Schierer 
study found that few organizations felt comfortable about their agency’s capacity for 
evaluation. Most of this was due to the fact that significant parts of the “evaluation type 
people” were downsized, as they were not considered “production type employees”. So 
while GPRA has required us to develop performance/outcome-based measures, most 
organizations have lost the personnel resources to do this type of work. In their 
recommendations, they offered “the system must strengthen the evaluation skills of their 


workforce as the basic tenet to enhance the likelihood that the performance management 
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framework being institutionalized via GPRA will result in both improved program 


management and desired results”. 


In Paul O’Connell’s study of the new York Police department’s CompStat system 
(Using Performance data for Accountability, August 2001) he found a sophisticated 
performance measurement system that reorders an organization’s day-to-day operations, 
as well as its overall orientation toward its core mission and goals. CompStat is a metrics 
tool that gathers various stakeholder input. It is based upon the compilation, distribution 
and utilization of “real time” data in order to allow field managers to make more 
informed and effective decisions. The author awards praise to the CompStat model 
because it included many stakeholders in its development and allowed key stakeholders 
to recommend specific measurement tools that would motivate performance. It is widely 
recognized as a significant reason for the reduction in serious crime in the New York 
area. However, O’Connell cautions, “The CompStat process is not a panacea. It is not a 
magical cure-all that will transform a poorly run and inefficient organization into a model 
of public service excellence. Rather, it is an additional tool that can be used to enhance 
performance by means of careful measurement and planning and effective allocation of 
resources”. The lesson to learn here is to identify all of the key stakeholders and allow 
them to recommend metrics that not only satisfy the strategic direction of the 


organization, but also motivates performance and awards successes. 


Frederickson (The Potential of the Government Performance and Results Act as a 
Tool to manage Third-Party Government) says, “GPRA seeks to improve federal 
agency’s efficiency and effectiveness”. GPRA posits that federal performance 
shortcomings are primarily managerial, specifically attributable to poorly articulated 
missions and inadequate performance information. Even if we better define our missions 
and track performance to that mission, our efforts can be curtailed by the fact that the 
third parties (customers, Congress, taxpayers) with whom we partner to deliver public 
services are not uniform in either kind of responsibility, or in other words, they do not 
share our values or mission. Frederickson’s research also recommended that when 
developing performance goals, agencies should make clear their role in the delivery of 
public service, and should use GPRA not only as a means to communicate their 


performance, but also to communicate constraints that inhibit their performance. It 
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would be interesting to report to Congress (a requirement of GPRA is to coordinate with 
key stakeholders) that they are one of the major inhibitors to your agency in achieving 
your performance goals. Frederickson also provided a summary of the recurring themes 
that the Government has encountered in previous attempts to implement a GPRA like 


system: 
These themes include: 


- It is difficult to reach agreement on goals and to find adequate 
measures to determine the attainment of goals even if there is 
agreement on what those goals should be; 


- Managers often turn to activity and output measures as proxies for 
outcome measures; 


- Performance measurement and budgeting represent the 
superimposition of a managerial structure on a political process; 


- While performance information has not proven useful for 
appropriators, it has shown some promise for management 
decisions. 


Given the previous attempts at reforms similar to GPRA, one might expect a 
similar result. However, Frederickson contends that all other reforms were done via the 
executive branch, while GPRA originated in the legislative or oversight branch. In 
addition, GPRA carries the force of law. Even if agencies thought it was another fad, 
they are legally bound to comply with its requirements. Frederickson’s research 
answered the question on how some agencies are implementing GPRA. He found, based 
on a preliminary review of the FY 2001 performance plans, with their emphasis on 
activity measures, rarely did an activity answer the question of Are these measures 
adequate to evaluate the achievement of program objectives? However, he points out, 
given the methodological and logistical problems associated with measuring a public 
program’s performance, it is not difficult to understand why the majority of and agencies’ 


measures reflect inputs and activities. 


So just by making it mandated by law does not provide us a solution to the 
problem. As the literature review has indicated, we are spending a lot of time trying to 
figure out how to comply with GPRA despite the fact it has been a law for 10 years. 


Agencies are constantly wrestling with contradictory Congressional messages. On one 
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hand you are held accountable for your results. On the other hand, you are constantly 
being pushed to devolve more responsibilities to lower levels of government or 
administer them through private entities and hold them responsible. Probably the best 
lesson to learn from Fredrickson’s research is that GPRA-driven reform efforts can 
accomplish reform only to the extent that they do not conflict with authorizing law and 
other existing laws. In his research he found that all of the agencies he was studying 
were impeded in their efforts to achieve their goals and collect data to determine the level 


of goal achievement by existing law. 


Leading organizations use their performance management systems to achieve 
results, accelerate change, and facilitate communication throughout the year so that 
discussion about individual and organizational performance are integrated and ongoing. 
Thus, an effective performance management system can be a strategic tool for 
organizations to drive internal change (efficiency) and achieve external results 
(effectiveness) (GAO 02 —966 Result Oriented Culture). GAO found that performance 


management systems in leading organizations typically seek to achieve three key 


objectives: 

1. Provide candid and constructive feedback to help individual 
employees maximize their contribution and _ potential in 
understanding and realizing the goals and objectives of the agency. 

De Provide management with the objectives and _ fact-based 
information it needs to reward top performers. 

3. Provides the necessary information and documentation to deal with 


poor performance. 
Performance management has to start form the top in order for the system to operate as 
designed. Recognizing this, the Office of Personnel Management (OPM) amended 
regulations on governing performance appraisals for Senior Executives requiring 
agencies to establish performance management systems that: 
I Hold senior executives accountable for their individual and 


organizational performance by linking performance management 
with the results-oriented goals of the GPRA. 


2 Evaluate senior executive performance using measures that 
balance organizational results with customer satisfaction, 
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employee perspectives, and any other measures agencies decide 
are appropriate. 


3: Use performance results as a basis for pay, awards, and other 
personnel decisions. 


Agencies were to establish these performance management systems by their 2001 


senior executive performance appraisal cycle. 


GAO found that most agencies have taken the first step in developing an initial set 
of expectations for senior executives to address in their individual performance plans that 
are intended to balance accountability for organizational results with a focus on customer 
satisfaction and a consideration of employee perspective. In their report to Congress 
(GAO 02 —966 Result Oriented Culture; Using Balanced Expectations to Manage Senior 
Executive Performance), GAO summarized the following: 

There is a growing recognition, driven by a variety of worldwide trends 

and pressing long-term fiscal challenges, that the federal government is on 

the brink of an enormous transformation in what the Government does, 

how it does business and in some cases who does the Government’s 

business. Ultimately successful organizations understand that they must 

often change their culture to successfully transform themselves, and that 

such a change starts with top leadership. Senior executive performance 

and accountability for change management will therefore be critical to the 

success of the federal government’s transformation. A_ specific 

performance expectation to lead and facilitate change could be a critical 


element as agencies transform themselves to succeed in an environment 
that is more results-oriented, less hierarchical and more integrated. 


The lesson to learn here is that there will be many influences involved in 
determining an activity’s performance measurement system, some of which will be 
political and some that may possibly be selfish. 

The Veteran’s Benefits Administration (VBA) adopted a balanced scorecard 
approach in fiscal year 1999 as a strategic management tool to drive organizational 
change, provide feedback to employees on measures they can influence, link performance 
appraisal and reward systems to performance measures, and provide incentives for 
managers to work as teams in meeting performance measures. Its scorecard included 
measures for accuracy, speed and timeliness, unit cost, customer satisfaction, and 


employee development and satisfaction. VBA incorporated these measures in the 
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performance appraisals for senior executives in their regional offices where the majority 
of senior executives are located. They have experienced some success with this 
approach. Later on in this chapter, the balanced scorecard approach will be analyzed in 
detail as an approach to achieve performance management success (GAO 02 —966 Result 
Oriented Culture). 

Effective performance management systems seek to provide candid and 
constructive feedback to help individual employees maximize their potential in 
understanding and realizing organizational goals and objectives and help reward top 
performers. In recognizing the importance of creating a results-oriented culture in federal 
agencies, Congress is considering legislative proposals to, among other things, focus 
attention on the impact poor performance can have on the effectiveness of an 
organization and require agencies to have chief human capital officers to select, develop 
and manage a productive high-quality workforce. (GAO 02-862 Results-Oriented 
Culture; Insights for U.S. Agencies from other Countries’ Performance Management 
Initiatives) However, this still does not answer the question of what you can do about 
poor performers. Until the Government relaxes their performance appraisal system to be 
more in line with the private sector, the Government will continue to experience the poor 
performance problem with limited to no recourse. Legislative action has to take the place 
of discussing what we do with poor performers. 


C, ORGANIZATIONAL CONFIGURATION MODEL 


Professor Nancy Roberts has done extensive research in the area of 
Organizational efficiency and effectiveness. | Robert’s perspective is that all 
Organizations are configured with attributes toward efficiency and effectiveness. The 
configuration approach is a “holistic stance” and assumes that the parts of an organization 
“take their meaning from the whole and cannot be understood in isolation”. The 
configurations represent a clustering of organizational attributes (e.g., environment, 
strategy, structure, culture, beliefs, processes) that fall into coherent patterns. The 
exercise of configuring an organization first starts by identifying the basic two 


dimensions of organizational performance — efficiency and effectiveness. 
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1. What Is the Difference Between Efficiency and Effectiveness and Why 
Should One strive for Both? 


Efficiency refers to the capacity to produce results with the minimum expenditure 
of time, money, or materials (Websters 1971, 725). Efficiency thus focuses on the input- 
output ratio. To be efficient is to do things well, to attend to the internal organization by 
refining, formalizing, and elaborating on existing knowledge and making short-term 
improvements. Effectiveness on the other hand, is defined as “productive of results” 
(Websters 1971, 724). The focus is on doing the right thing and that is determined by 
outcome attainment. It comes from an understanding and interpretation of the exterior 
environment and what outcomes are required (e.g., Customer satisfaction, goal 


achievement, negotiated savings). 


As discussed above, both efficiency and effectiveness play an important role in 
organizational performance, yet in the competition for resources, each can interfere with 
the other. Efficiency depends on focus, precision, repetition, analysis, discipline and 
control. Effectiveness, on the other hand, relies on experimentation, novelty and 
loosened control. While effectiveness thrives on experimentation, efficiency attempts to 
drive out any uncertainties. Consequently, public managers need to make choices 
between the level of efficiency and effectiveness they intend to pursue. Figure 1 
illustrates the four possible combinations, identified as configurations or ideal types that 
can be theoretically derived when public sector managers seek different levels of 
efficiency (horizontal dimensions) and different levels of effectiveness (vertical 


dimension). 


Efficiency looks at internal operations, while effectiveness is viewed as how well 
are we performing by the external environment (stakeholders). An Organization can be 
considered quite effective by a customer, yet be significantly inefficient when you 


examine their internal operating procedures. 


So why do we need to strive for both? Simply stated, we are required to. As 
stated previously in this chapter “NAVSUP has embarked on a journey to better structure 
and align the organization to most effectively and efficiently deliver combat capability 


through logistics”. In addition to Navy transformation, the goal of the GPRA is to 
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measure us by our outcomes and demonstrate to the taxpayer that we are an efficient and 


effective organization. 


HIGH 
ADAPTIVE GENERATIVE 
EFFECTIVENESS 
RESPONSIVE DIRECTIVE 
LOW 





HIGH 


LOW 
EFFICIENCY 


Figure 1. | Organizational Configurations 


2. Directive Configuration 


The Directive Configuration promotes optimal efficiency with less attention 
devoted to effectiveness. In this configuration, public managers pursue efficiency by 
running their organizations like well-oiled machines. They focus on maintaining internal 
order, thus attempting as much as possible to cut the organization off from any disruptive 
external influences. Managers insist on a common direction to set goals and objectives in 
order to keep members accountable. They require formalized jobs and standardized work 


to maintain an orderly coordinated activity. They oversee uniform policies that cover 
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rights and duties, promotions based on competence and merit, and an impersonal 
relationship with the workforce to ensure the smooth flow of work. They use budget and 
operational controls to monitor actions and most organizational change is geared to 
improve operations. Since change can disrupt orderly routines, they minimize it unless it 
is deemed essential. When forced to make a change, managers take on the role of 
strategic planner and driver of the organizational system by using top-down directives to 
modify standard operating procedures. The management direction setting and planning is 
a top-down approach that sets goals and strategy based on increasing efficiency. The role 
of the organizational manager is a planner/controller type and the role of the employee is 
“soldier” and order taker. Discretion and flexibility are minimal in deference to the 
norms of order, stability, and respect for higher authority. 


3. Responsive Configuration 


The Responsive Configuration relieves the tension between efficiency and 
effectiveness by ignoring both dimensions. What this means in practice is that public 
managers do not push their organizations toward effectiveness or efficiency, nor do they 
make much effort to reconcile the competing demands between the two. This is crisis 
management at its best as managers muddle through the decision making process. We 
see this type of organization typically in the political minefields. In their attempts to be 
responsive to external political forces, they are forced to make a choice between crisis 
manager (douse political fires when they erupt to accommodate contentious stakeholders) 
and power brokers (join in the fray by building a base of power and develop strategies to 
protect organizational interests). Whether they choose the role of crisis manager or 
broker of power organizational policies are the result of responses made to competing 
demands made in the political arena. Research evidence to support the Responsive 
Configuration comes from many sources. In the business literature, organizations falling 
into this configuration are often treated as reactors—poor performers—because they are 
considered neither effective nor efficient. Alternatively, in a highly regulated 
environment organizations falling into this configuration are viewed as good performers. 
The Responsive Configuration is viewed as an ideal type in a political economy that 
requires organizations to be accountable to the public by anticipating and responding to 
its concerns. Responsiveness, not efficiency or effectiveness, becomes the primary driver 
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under these circumstances. Whether public managers view there roles as crisis managers, 
who strive to accommodate political interests or whether they see themselves as power 
brokers who promote their preferred interests, responsiveness becomes the mantel by 
which they are measured. 


4. Adapter Configuration 


The Adapter Configuration seeks to optimize organizational effectiveness, 
downplaying interest in efficiency. Public managers achieve organizational efficiency by 
adapting to the external environment and meeting customer needs. They decentralize 
decision making so it rests on the shoulder of members who are closest to customers, 
substituting customers for bosses as the basis of their actions. They rely on the 
employee’s up-to-date knowledge of the industry and customer demands to provide 
insightful and innovative ideas to meet customer expectations. These types of 
organizations reward risk takers and innovative ideas. Organizations are decentralized 
and focus on collaborative projects and cross-disciplinary teams. Employees are given a 
general vision of the future, rather than specific goals and objectives, in order to inspire 
them to search for new ideas rather than a planned plotted course. Flexibility, creativity, 
exploration and experimentation are far more important to managers than rigid adherence 
to internal order and control. Research and Development organizations are typical of 
those in the Adaptive Configuration. National Aeronautics and Space Agency (NASA) 
during the 1960s is a good example of this organizational configuration. Given a very 
general mandate by President Kennedy to land a man on the moon within the decade, 
NASA’s goals for how to do it were neither fixed nor precisely determined. Addressing 
how to meet this mandate, required exploration, experimentation and learning from 
experience. No one had a formula for what had to be accomplished. NASA developed 
an agile organization where innovation would be more important than stability. As 
NASA’ administrator, James Webb’s credo was “You have to have a system that permits, 
encourages, and even forces flexibility and adaptive innovation on the part of those 
people—including the executives—who are responsible for its various elements”. Thus 
NASA went through various organizational changes to ensure flexibility and to induce a 
state of “organic flux”. The efforts produced a decentralized system reliant on program 
offices and field center for project management. The intention of senior management 
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was to give the centers what they needed to get the job done and field offices were given 
the authority to act independently when the situation warranted it. Although managers 
were expected to be knowledgeable in sound management doctrine and practice, they 
were also required to do a job without exact definition of what it was or how it should be 
done. There was no precedent to which a manager could examine. He was on his own to 
figure out a new or different solution. Maneuverability of the whole and the varied parts 
was essential for success. As Webb admitted, “we accepted a large degree of 
organizational instability in order to achieve the necessary degree of maneuverability.” 


35 Generative Configuration 


The Generative Configuration demands optimal efficiency and effectiveness. In 
this configuration, managers strive for both as they search for ways to reconcile the 
competing expectations emanating from the two dimensions. Central to operating in this 
type of configuration is the reliance on stakeholder collaboration. This is the first time 
we see the mention of stakeholder collaboration in our configuration models. Usually we 
are pitted against various stakeholders often with differing interests than ours. The 
generative organization relies on collaboration with the stakeholders to achieve both 
organizational efficiency and effectiveness. Efficiency derives from the networked 
stakeholders working together in the pursuit of common means. Effectiveness derives 
from the networked stakeholders working together in the pursuit of common ends. 
Stakeholder collaboration, the key element to a generative organization, takes on many 
forms, including multi-functional teams, partnerships, joint ventures, and alliances. 
Stakeholders may be internal or external to the organizations. However, collaborations 
usually require participants to come together quickly to exploit some opportunity or solve 
some problems that each acting independently could not address. The point is to leverage 
intellect and promote peoples generative learning—learning that develops their capacity 
to create new solutions to old problems rather than settle for adaptive learning that only 
prepares them for coping. The intent of generative learning is to open up new ways at 
looking at the world and to encourage deeper understanding of a system and its 
underlying dynamics. Finding public organizations that exemplify all the characteristics 
of the Generative Organization proves to be more difficult than it was for the first three 
configurations. There is evidence that some organizations that exemplify some traits of a 
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Generative Organization, but on a whole would not be classified as generative. There 
are: stakeholder dialogues and deliberations to craft public policy and organizational 
direction; learning strategies to review organizational procedures and chart new courses 
of action; measures of outputs and outcomes to track and improve organizational 
performance; dedications to customer service and offering only those products and 
services that add value. While lacking a true holistic generative organization, the many 
aspects that define a learning organization exist in the some public organizations today. 


6. Why is the Configuration Approach Important in Determining the 
Efficiency and Effectiveness of a Field Contracting Office? 


In Chapter 12 of Nancy Robert’s book, the configuration approach to 
management has provoked interest in the part of practitioners and scholars, and support 
continues to grow as researchers test the various configurations in different organizations 
and environments. The four configurations simplify the management jungle of competing 
theories and models for public managers and at the same time, raise the following 
fundamental questions: How efficient are we and how efficient must we become to 
reduce costs and conserve resources? How effective are we in meeting customer needs in 
a changing environment? How responsive are we to stakeholder issues and how 


responsive must we be, given competing political interests? 


In taking a configuration approach, the component parts of each ideal 
configuration “fit” together. Fit reveals the extent to which organizational components 
are mutually reinforcing and compatible. When the components fit and are patterned 
after a configurations ideal, the organization is expected to be a high performer; 
deviations from ideal types do not perform as well. This is an important concept to 
understand, especially as it relates to change. Managers must understand how the change 
may effect the “fit” to their organization. For example, wholesale adoption of techniques 
because they are promoted as “good business practice” from the private sector, without 
consideration of organizational fit, risks misaligning the organization and it component 


parts, which runs the risk of reducing rather than improving organizational performance. 


While the Generative Configuration is evolving and has not been found in its 
entirety as a separate configuration, it captures an important element—the network 


structure—that has been missing from traditional analysis of this area. Both the 
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Responsive and the Generative configurations capture characteristics particularly relevant 
in the public arena. Each acknowledges the importance of stakeholders to the 
organization’s performance: the Responsive Configuration assumes a competitive model 
of stakeholder relations, while the Generative Configuration assumes a collaborative 
model of stakeholder relations. Two of the ideal types (Directive and Adaptive) have 
been demonstrated to be high performers, and the third (Responsive) a good performer in 
a regulated environment. Therefore, we find there are multiple paths to high 
organizational performance and “one size does not fit all” and one configuration is “not 


better than the other’. 


High performing organizations tend to position themselves as ideal types at either 
end of the efficiency/effectiveness continuum. Other organizations have been found at 
varying points along either axis. There are many hybrid organizations found on the 
configuration model. These types of organizations have dual focus; it seeks the 
flexibility inherent in the Adaptive Organization, and the control and order characteristic 
of the Directive Configuration. Many public organizations combine the efficiency of 
machine bureaus with the adaptations and innovations that professional bring from their 
fields. This allows organizations to create subunits that specialize in effectiveness 
through innovation and change and customer building, while another subunit can mine 
traditional products and services to achieve greater efficiency. Thus the overall corporate 
entity can benefit because it combines subunits striving for either effectiveness or 


efficiency. 


In order to determine the effectiveness and efficiency of any organization, it is 
imperative that you first determine where you find yourself on the organizational 
configuration model. After you have done this, you can plot a course of action and 
management strategy to achieve the organizational goals. At this point you may be able 


to offset the competing interests of the efficiency vs. effectiveness paradox. 


That’s why it is imperative that contracting offices understand how they are 
configured organizationally so that they can make tradeoffs on efficiency and 
effectiveness. In other words, why strive for effectiveness if you are strictly a Directive 


Organization. 
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Te Where Do Most Field Contracting Offices Fit in the Organizational 
Configuration Model? 


HIGH 


FCO 


EFFECTIVENESS 


RESPONSIVE 





LOW 


Low HIGH 


EFFICIENCY 


Figure 2. Field Contracting Officer (FCO) Organizational Configuration 


The researcher took an informal survey as well as examined several field 
contracting offices mission statement and strategic plans to determine where they fit on 
the Organizational configuration. As depicted in Figure 2, most see themselves as a 
hybrid organization. Continuously striving for customer satisfaction (effectiveness) and 
supporting the war-fighter while offsetting that with ways to reduce infrastructure and 
cycle time (efficiency). Most Field Contracting Offices find themselves being graded on 
how effective they are in meeting both customer needs and the multiple stakeholder 
interests. When asked how they balance efficiency with effectiveness most mentioned 


the balanced scorecard as an approach to measuring not just effectiveness and efficiency, 


26 


but also the overall performance of the entire organization in how it meets its mission and 
supports its employees. The researcher now will examine the balanced scorecard 
approach and how it relates to the organizational configuration model. In addition, the 
researcher will provide some success stories of the balanced scorecard approach and how 
it may help in developing an organizational; model for evaluating efficiency and 
effectiveness. 


D. THE BALANCED SCORECARD MANAGEMENT SYSTEM 


In John Gray’s thesis (Measuring Satisfaction in the Program Manager — 
Procuring Contracting Officer Relationship, December 1997), the balanced scorecard 
management systems allows managers to look at their business from four important 
perspectives (financial, internal business, learning and growth, and customer). The four 
perspectives of the scorecard take into account the duality of short- and long- term 
objectives, desired outcomes and performance constraints, and objective and subjective 
measures. This system is useful because it recognizes that executives do not rely on one 
performance measure to manage an organization. Instead it offers managers a balanced 


presentation of both financial and operational matters. 


The financial perspective included performance measures, which indicate whether 
“the organization’s strategy, implementation, and execution are contributing to bottom- 
line improvement” (Kaplan &Norton, 1992, p.77). The internal business perspective 
provided “measures which focus on the internal processes that will have the greatest 
impact on customer satisfaction and achieving an organization’s financial objective” 
(Kaplan & Norton, 1996, p.27). The learning and growth perspective identifies the 
“infrastructure that the organization must build to create long-term growth and 
improvement. This is the rationale for significant investment in reskilling employees, in 
information technology and systems, and enhanced organizational procedures” (Kaplan 
& Norton, 1996, p.12). The customer perspective comprises “generic measures of the 
successful outcomes from a well formulated and implemented strategy. The core 
outcome measures include customer satisfaction, customer retention, new customer 
acquisition, customer profitability, and market and account share in targeted segments” 
(Kaplan & Norton, 1996, p.26). To utilize the Balanced Scorecard, organizations must 


first establish goals for each of the perspectives, and then translate these goals into 
a7 


specific performance measures. (Kaplan & Norton, 1992) The major advantage of the 
Balanced Scorecard management system, according to Litman And Wheeler (1997) is 
that it ensures that no organizational processes are ignored and that all types of 
performance measures are examined by organizational decision-makers to present a clear 


picture of the status of the organization. 


The Procurement Task Force formed by the President’s Management Council has 
recommended that agencies utilize the Balanced Scorecard management system. Toward 
this end, the Procurement Task Force has identified four major goals for the procurement 
system: quality, timeliness, price and productivity, which organizations can utilize to 
build their scorecard systems. Figure 3 represents a graphical depiction of the diamond 


shaped balanced scorecard (www. isixsigma.com) 


. How do we look to 
eres AT our shareholders? 


GOALS | MEASURES 








What must 
we excel at? 


Business Pers pective 


GOALS | MEASURES 






Learning Perspective 


GOALS | MEASURES 


Figure 3. | The Balanced Scorecard (From www.isixsigma.com) 


Many changes have happened to the balanced scorecard since its inception in 

1992. Although the concepts behind the balanced scorecard are fairly simple to 

understand, they are difficult to implement (Graham Brown’s, the Winning Score, 2000). 
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Graham Brown postulates that measurement is very difficult in organizations because it is 
not an exact science with hard rules and predictable interrelationships between variables. 
People run organizations and their actions are sometimes inherently unpredictable. 
Hence, a measure that made perfect sense at one time might now seem laughable when 
one sees the behavior it drove in the organization. The balanced scorecard is all about 
picking the right metrics. One must be careful about how they select metrics. When you 
focus on improving one metric, two other metrics might decline. We find samples of this 
in the decade of the 80s and 90s. The 80s was dedicated to improving productivity, as 
the perception was that the work force was lazy and unproductive. Productivity 
improved as we approached the close of the decade, but quality worsened. The decade of 
the 90s focused on the quality movement, which in many cases lowered financial results. 
As we enter the 21" century, the new mantra for organizational performance is balance, 
hence the balance scorecard management system. 

In his introduction, Graham Brown extols the virtues of measuring what matters. 
He cites “if you find that the chart of performance data you have been examining every 
month for the past year is merely confirming what you already knew, you should 
consider dropping that metric from your scored card”. He provides the following tip 
concerning measurement, “Many things in life are done well without requiring formal 
data. Measure only those factors that will provide you with information you didn’t know 
already and help you improve performance or make better decisions”. The balanced 
scorecard is a dynamic performance measurement tool and has helped many 
organizations, both public and private, but it requires much work and expertise to 
properly implement it. There are many mistakes made in determining what indeed are 
appropriate measures. Graham Brown provides a list of the top 10 measurement 


mistakes. He bases this on his years of consultation with both public and private 


organizations: 
1. Tracking Output/Outcome Metrics that cannot be influenced or 
controlled. 
2, Gathering data that tells you what you already know. 
Gathering data for its own sake. 
4. Relying heavily on Customer Satisfaction Surveys. 
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5 Executives focusing on detailed metrics. 

6 Measures are not linked to the strategic plan. 

{2 Failing to define practical correlations between key metrics. 
8 Reporting data that is difficult to read and analyze. 

9 “Superstitious” process metrics. 


10. Measures that drive the wrong performance. 


The challenge is choosing the right metric for the right reason for the organization, 
keeping in mind the strategic goals and objectives. The Procurement Executive Council 
(PEC) recommends using a cost to spend model as a way to maximize the efficiency of 
the procurement office relative to purchasing cost (PEC 2001). Later in Chapter II, the 
researcher will analyze the different types of metrics currently gathered for a typical field 
contracting organization. 

An important and difficult decision that needs to be made right off the bat is 
where to begin the scorecard project. One simple answer is to begin at the top. 
Conceptually this makes a lot of sense because it is easier to develop lower level 
scorecards that link together if the metrics for the CEO have been defined first (Graham 
Brown, 2000 p.232). Many companies and Government agencies have implemented the 
scorecard at many different levels of the organization. The point to be made is that it is 
fine to start the Scorecard initiative at any level of the organization. Starting at the top is 
probably the easiest in some ways, but politically this is often just not possible. Often, 
the best way to sell the CEO or Commanding Officer on the concept is to show it 
working well in key parts of the business or organization (Graham Brown, 2000 p.233). 


E. CHAPTER SUMMARY 


This chapter included an overview of a field contracting office and provided a 
basic understanding of where a field contracting office fit in the overall Navy Contracting 
function. Also, it included a review of the Navy Transformation effort that and how that 
impacts a field contracting office. The main thrust of the NAVSUP transformation is “to 
better structure and align the organization to most effectively and efficiently deliver 
combat capability through logistics”. With that being the mantra, it is imperative for a 


field contracting office to ensure that they are as efficient and effective as they can be. 
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Next, the chapter focused on the literature review that was conducted concerning 
efficiency and effectiveness. Based on this review, the researcher found many examples 
of the importance associated with obtaining efficiency and effectiveness. The GPRA was 
the main catalyst for focusing Government agencies on measuring outcomes and results 
and moving away from just measuring outputs. The literature review included an 
analysis of many GAO studies as well as endowment studies that examined how 


organizations were attempting to satisfy the requirements of the GPRA. 


The third area the chapter examined was the organizational configuration model 
developed by Nancy Roberts. Robert’s model, which describes different types of 
organizations (Directive, Responsive, Adaptive, Generative), gives one framework to 
understand where a field contracting office fits in this model. By understanding the “fit” 
of an organization, you can better strategize how to achieve the balance between 


efficiency and effectiveness. 


Finally, the chapter examined the balanced scorecard approach, which many 
organizations are currently using to balance all aspects of their operations. The balanced 
scorecard approach gives an organization a picture of how they are performing as an 
overall entity and looks at all parts of the organization (financial, internal operations, 
customers and employees). The researcher found that many organizations use the 


balance scorecard approach to maximize both efficiency and effectiveness. 


Si 
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ll. DATA PRESENTATION AND ANALYSIS 


A. NAVY STUDY’S 

The researcher found a summary of a series of studies performed by the Navy to 
evaluate the overall performance of a Contracting Office. The summary was provided by 
a consultant contractor, Booz-Allen & Hamilton (BAH), in response to another study 
they are conducting for the Navy titled, “Establishing the Effectiveness and Efficiency of 
the Department of Navy’s Material Establishment”. The researcher was not able to 
obtain a copy of the various studies cited in the BAH report, however, BAH’s summaries 
provided the researcher with a starting point to analyze how the Navy has historically 
looked at the efficiency and effectiveness of the Contracting function. The studies did 
not limit themselves to a field contracting office, in fact, most of the studies were 
performed to determine the most effective way to organize the contracting function in the 
Systems Commands or the Navy overall. Unfortunately most of the studies to date draw 
on anecdotal evidence and therefore, they only generally address and hypothesize on the 
efficiency and effectiveness of a contracting office. However, they did address areas 
concerning contracting office roles and responsibilities as well as overall efficiencies that 
can be gained by consolidating offices. There are some aspects to the studies that did 
address cost and effectiveness of operations. Efficiency was addressed on an enterprise- 
wide basis. The studies provide a background into what the Navy leadership considers 
relevant when assessing the effectiveness of its contracting function. As you will see in 
the analysis of the studies, there still seems to remain a perception that all contracting 
professional (also known as “1102s”) are created equally with the same skill mix to 
acquire any type of supply or service. 

In addition, the researcher found a private sector study, sponsored by the National 
Association of Purchasing Managers (NAPM), that analyzed what measures are 
considered useful to a CEO when assessing the overall effectiveness of his purchasing 
operations. Also, another private study, sponsored by NAPM, analyzed the roles and 
responsibilities of Purchasing Office and how it relates to the overall strategy of the 


company. 
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The remaining portion of this chapter will show the name of the study, purpose, 


and the Analysis/Metric Used. Then the findings or recommendations will be provided 


with an overall analysis of the study. The researcher will then analyze NAVSUP and 


FISC Norfolk’s current metrics and review what the Defense Contract Management 























Agency (DCMA) measures. 
NAME DATE PURPOSE ANALYSIS/METRIC 
USED 
NAVSEA 02 HQ 12/17/91 To consider the consolidation of 1. Consolidate 
Contracting contracting organization three alternatives: 
Consolidation Study functions. 
- Status Quo 
Is consolidation feasible? - Naval Systems 
Contracting Command 
Could it continue to provide with HCA authority, with 
adequate support to customers? contracts personnel co- 
located with customers. 
Would it result in efficiency - Combine Hardware 
savings through personnel Systems Commands into 
reductions? one SYSCOM with the 
resulting contract 
organization moved to 
one site 
2. Considered pros 
and cons of each 
alternative based on 
anecdotal evidence. 
3. Data used was 
primarily personnel 
headcount, and projected 
reductions that were 
anecdotal. 
Table 1 NAVSEA 02 HQ contracting Consolidation Study 
1. Recommendations and Findings 
1. Recommended maintaining the status quo. 
2 Efficiencies (personnel reductions gained only through consolidation of 


support and front office functions. 


3 Unclear that any efficiency savings were gained in the buying function 


(number of touch personnel assumed to be consistent with workload). 
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2. Analysis 

While this study provides a historical contracting perspective of consolidation that 
spans back to 1991, it lacks actual data analysis and relies on anecdotal evidence. 
Finding 3 cited above gives the impression that the number of people is assumed to be 
consistent with workload. This is not always true in the contracting organizations 
interviewed as well as what has historically transpired at the researcher’s contracting 
activity. In addition, the study does not define what workload actually means. Is it the 
number of purchase requisition on your desk? The number of delivery orders processed? 
Or does it include the unmeasured workload of acquisition planning and market research? 
The different complexities of the workload must be examined in order to make the 
statement that the number of personnel is consistent with the workload. One complex 
service acquisition could be considered equivalent to 100 delivery orders as they relate to 


personnel and workload. 








NAME DATE PURPOSE ANALYSIS/METRIC 
USED 
ASN RD&A (ABM) | 6/28/95 Provide assessment of how Considered three alternatives: 
Study on Contracting contracting could be potentially 
Functional input organized under a single 1. Status Quo 


SYSCOM, including: 
2. Examine a CEO 


- Role of Contracting in the Navy Organization. (Locates 
functional communities 
- Assumptions and precepts of with customers as well as 
successful contracting function. along product lines in an 


IPT format) 
- Recommend an organizational 
structure (as well as analysis of 


alternatives considered) 3. Examined a Matrix 
organization, which is 
- Investigate cultural norm of the similar to a CEO but 
contracting community. without a Board of 
Directors or product line 
affiliation. 


. Considered pros and cons of 
each alternative based on 
anecdotal evidence. 




















Table2 ASN RD&A Report on Contracting Functional Input 
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3. Findings/Recommendations 


1. Contracting personnel evaluations must remain “stovepiped” to maintain 
independence. 
ys Current contracting organizations: 


Accommodates geographical dispersions easily. 
b. Gets the job done. 
C. Presents limited opportunity for consolidation of redundant 
activities (policy and management groups are in most activities). 
d. Does not facilitate inter-SYSCOM or inter-Command interaction. 
3. Recommended CEO Organization because it rationalizes redundant work 
across material establishment and provides a single point of contact for all 
acquisition matters. 
The following other alternatives were noted, but not considered, including: 


- Centralization at one site. Not considered because of degradation in 
service to customer. 

- Single SYSCOM with regional offices. Not considered because it creates 
redundant functions. 

- Product oriented contracting office. Not considered because it was too 
similar to CEO and matrix. 

4. Analysis 


This study presented a few interesting ideas on restructuring contracting on an 
enterprise-wide basis. However, once again it was based on anecdotal evidence with no 
real investigation of contracting office efficiencies by combining functions or 
products/services. All the analysis was based on a theoretical framework. This study 
begs the follow-up question of: Were there counterpart studies to this analysis to include 
consolidation of the financial or inventory function? In addition, the study recommended 
implementation of the CEO organization, yet the Navy for the most part completely 
ignored this alternative. It is interesting to note that one of the findings was that 
“contracting personnel evaluations must remain “stovepiped” to maintain independence”. 
The study then recommends the “CEO organization” as a way of combating this 
theoretical inefficiency. Much has changed since this study was performed in 1995. The 
advent of the Integrated Product Team (IPT) has changed the way the organization feels 


about the stove-piped nature of the contracting function. All the stakeholders in an IPT 
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bring a functional expertise to the team that allows early identification of problems and 


issues and considers all functional areas (financial, legal, contracting, engineering, and 


logistics) integral to the success of the acquisition. 


So to some degree in many 


organizations, the CEO philosophy has taken hold. However, in my interviews with 


other field contracting offices, IPTs are rarely officially formed because the participants 


involved are from different customer activities. 








NAME DATE PURPOSE ANALYSIS/METRIC 
USED 
ASN RD&A (ABM) | 1/22/98 Baseline current organizational Considered the following 
Navy Contracting structure. alternatives: 
Organizational 
Alignment Study Assess organizational 1. Transfer some contracting 
Team (NCOAST)) alternatives, including evaluation resources to fleet but 











of: 


- Top level costs 

- Responsiveness to customer 

- Advantages and 
Disadvantages 


Conduct analysis around the 
hypothesis that “smaller 
organizations could be 
consolidated with larger 
contracting organizations to yield 
labor dollar savings”. 


Conduct regional analysis of 
consolidation in Hampton Roads 
and Northwest Region of the 
United States. 


Survey contract customers to 
determine current satisfaction 





maintain current HCA 
placement. 


2. Transfer some contracting 
resources to fleet and 
centralize HCA placement. 


3. Transfer some contracting 
resources to fleet and 
delegate HCA to fleet. 


4. Transfer some contracting 
resources to fleet and 
delegate HCA to regional 
commanders. 


5. Consolidate all 
contracting functions into a 
single SYSCOM. 


6. Consolidate field 
activities into regional 
commands. 


7. Collected data through 
comprehensive data call to 
over 600 organizations and 
queried contracting 
customers via surveys 


Used the following evaluation 
criteria to evaluate alternatives: 
Responsiveness, Cost, 
Accountability, and Regions 





Table3 ASN RD&A Navy Contracting Organizational Alignment Study Team 


(NCOAST) 
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5. Findings/Recommendations 


The baseline organization consisted of 570 activities, 10,000 personnel, 1.3M 
actions, and $39B in obligations and $500M in direct labor cost. The study found that 
there was an organic cost savings potential of $83 to $199M (16-39% savings) by 
consolidating activities. The study also indicated that workload was declining, which 
would be expected given the reduction in the Defense budget. 

The Study observed the following about the status quo structure of the contracting 
function: 

- High customer satisfaction due to decentralization and dedicated 

resources. 

- Inhibits sharing of best practices within and among HCA’s. 

- Responsibility and accountability are not consistently placed with 

organizations responsible for the mission. 

- There is substantial duplication of effort. 

- Contracting missions are misaligned (ownership and management 

of contracting function is inconsistent with mission). 

The study also noted the following challenges associated with consolidation or 
regionalization: 

- Geographical proximity to customer. 

- Alignment of ownership and management of contracting resources. 

- Productivity and workload, cost, customer satisfaction, and 

accounting difficulties between an Operations and Maintenance 
Navy (O&MN) and Navy Working Capital Funds (NWCF) 
activities. 

The study found that no matter which organization structure selected, significant 
cost savings could be achieved by eliminating small activities and they cited a 1992 CNO 
study). They also found that customers considered the following important when 


discussing contracting personnel: 
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- Geographical proximity. 

- Centralized contracts for supplies. 

- Dedicated contracting support. 

- Use of Purchase Cards. 

While the study was probably the most comprehensive analysis of the contracting 
function navy-wide, it only recommended a few courses of action. 


1. Reduce the amount of activities with contracting authority. This 
will allow for infrastructure reduction and regionalization. This 
may also lead to increasing customer responsiveness by 
restructuring to better align resources to support mission functions. 
Increase Purchase Card authority to $100K. 

Conduct further analysis of consolidation of contracting operations 
in the Hampton Roads area. 

4. Establish a navy-wide database to track direct labor costs, 
overhead, actions, and obligations and other contracting functions 
to quantify efficiency and effectiveness and request contractor 
support to do further analysis to look at diversity of workload and 
its impact on customer satisfaction. 

Accelerate full implementation of SPS in order to increase 
efficiency and permit record/document sharing between activities. 


aired 


Nn 


6. Analysis 

The NCOAST study was probably one of the most comprehensive reviews of 
Contracting functions in Navy organizations. It consisted of data analysis that included 
actions, dollars, direct labor costs, indirect costs, mission funded organizations and Navy 
working capital fund organizations. The study also provided HCA responsibilities, along 
with source of responsibility and ability to delegate decision-making to lower levels. 
Some of the findings were surprising (centralized contracts for supplies). It is hard to 
understand why a customer would really care if the contracting office centralized 
contracts for suppliers. One would think that a customer’s main interest is getting what 
they need when they need it at reasonable prices. And others were obvious (reducing the 
amount of offices with contracting authority will reduce infrastructure costs). This action 
in and of itself will reduce infrastructure but it may increase costs in other parts of the 
enterprise and that’s why a thorough efficiency review or business case analysis must be 
performed before implementation of any action. While the study was comprehensive in 


its look at cost associated with the contracting function, it failed to look at complexity of 
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workload, and the system’s ability to consolidate work in a center of expertise concept. In 
addition, the recommendation to establish a navy-wide database to collect important 
contracting data seemed to be a reasonable recommendation that would gather real data, 
while providing a useful tool to help manage the Navy’s contracting community. 
Unfortunately, to the best of the researcher’s knowledge, this recommendation has not 
been implemented and the Navy to date still has not conducted an analysis of workload 
complexity and how it relates to improving overall efficiency and effectiveness. 

The NCOAST charter was to conduct an evaluation of existing contracting 
organizational structures, relationships and processes to identify alternatives that will 


enable the Navy to better support its customers (effectiveness), streamline processes 








(efficiency), reduce infrastructure (effectiveness) and eliminated inefficiencies 
(efficiency). The NCOAST team went to great lengths to analyze customer support and 
reduce infrastructure goals, but did not dedicate much effort in streamlining policies and 
eliminating inefficiencies. The significance of this study was its reliance on actual data 
and not anecdotal analysis. However, it also relied on much anecdotal evidence. For 
instance, a hypothesis was developed proposing that smaller contracting organizations 
could be consolidated with larger contracting organizations yielding labor dollar savings. 
A methodology was developed (based on this hypothesis) to mathematically represent 
labor dollar savings within the Navy contracting organizational structure. This 
methodology was based on a Center for Naval Analysis (CNA) study that theorized 20- 
50% labor savings based on consolidating existing organizations. The team used labor 
dollars from the Hampton Roads and Northwest regions to build a Navy-wide model. 
This hypothesis for projecting navy-wide labor dollar savings using the Hampton Roads 
and northwest region as a benchmark appear misleading as the majority of smaller 
activities are located in San Diego and Hampton Roads/Washington DC area. In 
addition, CNA only theorized a 20-50% savings, which first, is a large percentage 
difference and second, is not developed on factual data, just theorized savings. 

The NCOAST study did take a significant look at customer satisfaction. This 
measure of effectiveness seemed to weigh heavily in their recommendations as presented 


above. While the data analysis did not go deep enough on the efficiency or productivity 
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on contracting offices, it did provide enough background to allow further analysis of the 


contracting function. 























NAME DATE PURPOSE ANALYSIS/METRIC USED 
ASN RD&A (ABM) | 3/25/99 | Reduce contract cost to the Study identified savings opportunities that 
Navy by at least 10% and involved consolidation at 3 levels: 
The Hunt for 10%: deliver the same quality 
A Framework for product. - Consolidate common requirements 
Attacking Contract across HCA’s. 
Costs Put a detailed global strategic 
plan in place - Consolidate common contract 
support functions. 
Identify those areas in industry 
where this pressure would - Consolidate within HCA 
potentially derive the most 
savings. The process for evaluating and arriving at 
savings opportunities was not outlined. 
The contracting markets in which the 
Navy participates and approximate dollar 
amounts spent (FY 98) were identified 
along with the number of Navy 
contracting activities by region. 
Table4 ©ASN RD&A (ABM) The Hunt for 10%: A Framework for Attacking Contract 


Costs 


Te Findings/Recommendations 


The study identified a decrease in dollars spent on supplies and research and 


development contracts from 1988-1997, while service contract Dollars remained steady. 


In addition, the service to supply ratio more than doubled from 1988 — 1997 (1.e., service 


contracts went from 24% of contract dollars in 1988 to 40% in 1997, while supplies went 


from 58% to 42% in the same period). 


8. Analysis 


This study was performed as a preview to the next study. The working group 


provided a basic framework for the next team to further analyze potential savings to the 


Navy. The next study team reported out 25 days after this working group and their study 


is provided for review next. 


Al 








NAME 


DATE PURPOSE 


ANALYSIS/METRIC 
USED 








ASN RD&A (ABM) | 4/19/99 Reduce contract cost to the 


Navy by at least 10% and deliver 


Red Ink Rising: A the same quality product. 
Framework for 

Attacking Contract Put a detailed global strategic 
Costs plan in place 


Identify those areas in 
industry where this pressure 
would potentially derive the 
most savings. 











Analysis identifies and defines 
key characteristics for each market 
including commercial 
applicability, economies of scale, 
proprietary data, entry/exit barriers 
and legal barriers. 


Market share by industry 
classification analysis identifies the 
Navy’s share of spend (by dollar 
amount and percentage) in each 
industry/market it participates. 
Study relied on 1998 data. 





Table5 ASN RD&A (ABM) Red Ink Rising: A Framework for Attacking Contract Costs 


9. 


Findings/Recommendations 


The overall findings of the working group were as follows: 





The Navy interacts in 2 market cells within a 3 X 3 customer/seller 
matrix, few to few and many to many. 

An example of few to few is Research & Development (aircraft & 
airframes, ships, engines/turbines. 

An example of many to many is maintenance and repair of 
equipment, professional and management services, maintenance of 
real property, ADP and telecommunication services. 

There is no low hanging fruit or obvious opportunities for 
leveraging. Sellers are decreasing and future contract costs are 
being driven by current acquisition decisions. The group found 
that the ultimate responsibility for savings is at the working level. 


The study group recommended that the Navy institute market sector brokers to 


facilitate contracting strategy development for the procurement of common and related 


requirements at the appropriate level within the “corporate” Navy. Also, the team 


recommended that the Navy team with contractors to pursue reduced infrastructure and 


indirect cost rates at prime and subcontract levels and the navy should also pursue 


improved cash management of unexpended contract funds. 
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10. 


Analysis 


The idea of having a “Corporate Strategy” for the contracting and requirements 


function is a novel idea. By establishing market sector brokers or market managers, the 


Navy can begin to consolidate their requirements and allow the Contracting Office with 


the most expertise in that market to satisfy the requirement. In theory, we will have more 


leverage in dealing with the private sector and in addition, contracting offices can become 


much more efficient. By establishing “Centers of Excellence” for contracting that utilize 


the specific expertise of that contracting office, the Navy can ultimately reduce 


infrastructure costs across the Navy enterprise via either strategic sourcing or functional 


assessments to determine “expert” offices. This would also change buying habits, hold 


contract cost steady and provide leverage to private industry solutions. Unfortunately, 


there are no data sources identified in the study, so the researcher can’t determine where 


the study group received their information. 








cross command 
contracting 





Potential Savings from 








Recommend approach for 
cross-command contracting 


Identify other approaches for 
reducing contract costs. 





NAME DATE PURPOSE ANALYSIS/METRIC 
USED 
Center for Navy 3/09/00 Assess potential for cross- Analysis included an 
analysis Study command contracting evaluation of FY 99 obligations 
Briefing across product lines and 


SYSCOMS. 


Analysis also included a 
description of ongoing cost 
reduction initiatives 





Table 6 Center for Navy Analysis Study Briefing: Potential Savings From Cross- 


11. 


Command Contracting 


Findings/Recommendations: 


The overall findings of the working group were as follows: 


There is an overall lack of requirements visibility before award. 


There is a lack of Cross-Command initiatives. 
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- There are numerous initiatives underway, however, most are 
internal to Commands. 

- Lack of command-level incentives to pursue savings initiatives. 

- Data are not available to maximize opportunity. Either the 
information is not available within or among activities or the data 
systems are only good after the fact. 


The study group did not making any recommendations. However, they did 
provide an outline of “models of consolidation” for industry. Unfortunately, the 
researcher could not get a copy of this model and to the best of his knowledge; this model 


has not been piloted or implemented at any Navy Command. 
12. ~— Analysis 


While the study indicated that FY 99 data was used, it does not indicate the data 
source. It is difficult to determine exact product lines based on the current management 
information systems in the Navy. To the best of the researcher’s knowledge, most 
analysis of product lines or services is done via a study of the Federal Procurement Data 
System (FPDS). The FPDS is limited in that it only categorizes a product in a general 
Federal Supply Code (FSC). Without actually looking at the contract, it is difficult to 
determine if the supply or service is a candidate for consolidation. In addition, this 
analysis is all after the fact and provides us little benefit in forecasting future 
requirements or even forecasting consolidated requirements. More robust management 
information systems need to be implemented that tracks the requirement form initiation to 
completion. The system must be standardized so that all members of the acquisition 


community share a common definition of the product or service being acquired. 


On October 24, 2002, ASN RD&A (ABM) issued an executive direction to all 
Navy Systems Commands addressing the results of an Overhead study. The researcher 
found no evidence of who performed the study or what data they used in performing the 
study. The ASN directive inferred that a consolidation of overhead functions could save 
10% or roughly 60 Full Time Equivalents (FTEs) across the enterprise. The ASN 
directive provided a detailed look at Overhead Activities. The researcher recalls 
answering a data call in 2002 to NAVSUP concerning FTEs performing overhead duties. 


In theory, the consolidation of overhead type functions across the enterprise, makes 
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sense, after all, why do you need 10 policy directors. Historically we organize our 
activities in relationship to headquarters and there is a trickle down effect that is 
continuously happening. For example, ASN will issue a policy directive to all the 
Systems Commands, the Systems Commands will then put their spin on it (which is 
usually just reiterating what ASN has said) then the Field Contracting Office puts their 
spin on it and implements the policy or guidance. The enterprise can reduce their 
overheads by standardizing policy in one field office, whose responsibility it is for 
implementing throughout the entire enterprise, thereby eliminating redundancy. That is 
just one example of savings that can be achieved through consolidation. However, for 
success to be achieved standards have to be established and buy-in must be achieved 
across the enterprise. Without that each office will resort back to old ways of doing 
business as they perceive no value added in the revised structure. Without having 
knowledge of the actual study that ASN performed, the researcher will assume that actual 
data was analyzed in order to come up with the 60 FTE savings. The only concern the 
researcher has is in ASN’s definition of overhead activities. There are some overhead 
activities that are difficult to consolidate among offices, like bid offices, distribution and 
file room maintenance. 


B. A PRIVATE INDUSTRY STUDY ON MEASURES OF PURCHASING 
EFFECTIVENESS 


In 1997, Harold Fearon Ph.D., C.P.M and Bill Bales, C.P.M conducted a research 
project for the Center for Advanced Purchasing Studies (CAPS www.capsresearch.org) 
on the measures of purchasing effectiveness. The foreword of their study state 
“Measuring performance of the purchasing function continues to be a vexing problem. 
Hundreds of articles and monographs have been written on the subject over the past 


several years, often looking for a "magic formula." If there is one, it might include these 


steps: 
L. Review the objectives of the organization to determine the 
measurements needed. 
2 Search for measures that match with the goals. 
a Review with senior management, the chief purchasing officer’s 
(CPO’s) superior, and major internal customers to determine what 
they need. 
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4. Don’t over or under measure. 


Ds Recognize that what you measure today will differ from what will 
be needed tomorrow. 


The objectives of the study were to determine what measures are considered to be useful 
by Chief Purchasing Officers (CPOs), and Chief Executive Officers (CEOs), that 
measure are CPOs currently using and are there any established guidelines CPOs can use 
to select purchasing performance measures. 


1. Research Findings 


There was considerable variation in how CPOs viewed the purchasing operations, 
and in many cases, it all depended on the authority delegated to them by leadership. The 
research showed that in terms of what the CPOs considered to be the most valuable 


measures for assessing purchasing effectiveness was: 


1. Use of leverage through combining volumes. 
2. Accurate, timely, and efficient data collection. 
3: Price negotiations resulting in savings. 


But in terms of actual measure usage, the three highest-rated measures were: 


1. Price negotiations resulting in savings. 
2. Department budget versus actual expenditure. 
3: Past-delivery performance. 


The measures in the organization, managing supplier base, and overall performance 
evaluation areas simply are not receiving the same relative attention as those in 
developing supplier base, cost effectiveness, and systems utilization. Also, the CPO 


found that the five least important measures were: 


1. Percent of annual dollar volume delivered directly from supplier to 
user. 

2. Number of major problems solved within 90 days. 

3. Percent of suppliers empowered to input to design and 
specifications. 

4. Purchasing dollars moved from outsourcing to in-house. 


Number of supplier satisfaction surveys this year versus goal. 
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The CPOs’ ranking of measurement guidelines showed these four as the most important: 


1 Chief purchasing officers (CPOs) must constantly be on the alert to 
add new and appropriate measurements, and to delete unneeded 
ones. 

2 Measurements are primarily for the purchasing manager’s use in 


managing and monitoring the department. 


3: Good computer systems are essential in supporting the flexibility 
needed to change and adapt measurements. 


4. Measurements should include a trend analysis to indicate direction 
of progress. 


The CEOs/presidents, when surveyed on their need for purchasing measures, said the 
most-needed measures primarily were non-financial in nature. The five most-needed 
measures, in order, were: 

I. Quality of purchased items. 


2; Key supplier problems that could affect supply. 


3 Supplier delivery performance. 
4. Internal customer satisfaction. 
aK Purchase inventory dollars. 


The five least-needed measures, in order with the least-needed shown last, were: 


1. Purchases from minority-/female-owned firms. 

2. Purchasing department headcount. 

3. Actual price paid versus a published market basket index. 

4. Supplier assessment of purchasing performance. 

Di Purchasing administrative cost as percent of purchase dollars 


spent. 
The CEOs wanted a trend analysis on many of the measures, and the timing of their 
need for many of the measures varied. Purchasing should check with senior management 
to see what specific measures are needed, how frequently the information is needed, and 


in what form the information should be presented. 
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Zs 


Conclusions 


There are no "quick fixes" to the purchasing measurement problem. Each 


organization is unique and requires measures tailored to its current environment and 


people. However, the research recommended 12 guidelines that should be followed in 


establishing a measurement system: 


dh, 
2: 


Measures need to be designed for use at a point in time. 


Each organization has specific measurement needs at a given point 
in time. 


Measures should address financial results, supplier performance, 
computer systems, and internal practices and policies. 


Measures must change frequently. 

Trend analysis often is useful. 

Measures should not be overdone or underutilized. 
Measures are only tools. 

Benchmarking is a source of new ideas and measures. 
Senior management must see value in the measures used. 


Measures can show the effectiveness of purchasing, and identify 
areas needing improvement. 


Ensure the credibility of measures. 


Continuous improvement in purchasing depends on measurement. 


Analysis 


This study showed some significant dichotomies between what a Chief 


Purchasing Office considers important to the effectiveness of a purchasing office, 


compared to what a Chief Executive Officer finds relevant. CEOs are interested more in 


those measures, which focus on quality of performance, rather than the traditional 


quantitative/financial measures, such as headcount and cost to spend a dollar. CEOs 


were mostly interested in the quality of the supplies and the quality of the relationships 


purchasing personnel establish with both suppliers and internal customers. CPOs, on the 


other hand, is interested in the day-to-day operations of their office. It seems odd that 


CPOs feel that they are measured on the savings they accrue from price negotiations, 
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while the CEO really only cares about the quality of the product or service delivered. 
This contrast can many times lead to opposite results. By driving down the price for a 
product or service, the CPO may be reducing the quality received. On the other hand, 
ignoring cost of operations and only concentrating on supplier relationships could lead to 
significant inefficiencies and higher costs, which may all of a sudden become a very 
important measure to a CEO. The balancing act required by a CPO is a very challenging 
one. He/she must focus on how they are evaluated by their bosses and then develop 
operating procedures that will ensure efficient internal operations while satisfying the 
CEO and other internal customers. It should also be pointed out that CEOs and CPOs for 
most part are not required to be socially economical when entering into contracts. They 
obviously do not have the same goals and requirements that a public contracting office 


has in meeting small business and other socially driven programs. 


In research (Purchasing’s Organizational Roles and Responsibilities, Fearon, PhD 
and Leenders, DBA, CAPS 1996)) that attempted to define roles and responsibilities in 
purchasing offices, it showed that purchasing’s current involvement in major corporate 
activities shows outsourcing as the only activity with a moderate involvement. 
Particularly low were Government relations, international counter-trade/offset planning, 
marketing planning and corporate merger/acquisition/alliances. In addition, the same 
research study showed that the most common roles and responsibilities, in descending 
order of importance, of a purchasing office were: 

- Establishing policies and procedures. 

- Executing contracts for common requirements. 


- Participating in system-wide purchasing/supply personnel decisions and 
actions. 


- Developing supply systems. 


C. WHAT METRICS ARE CURRENTLY BEING USED TO EVALUATE 
CONTRACTING OFFICE PERFORMANCE AND WHAT ARE THE 
LIMITATIONS OF THOSE METRICS? 


The following is a list of the current Contracting metrics that NAVSUP is 
gathering and presenting to the Chief of the Navy Supply Corps on a monthly basis as 
part of the Chief’s overall “dashboard” (see the Balanced Scorecard presented in Chapter 
I): 
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Number of Reverse Auctions conducted and saving achieved. 
Percentage of Performance Based Service Contracts (PBSC). 
Percentage of work Competed. 


Contract Closeout and the number of over-aged contracts. 


ae ee 


Navy and Marine Corps (NMCI) — Meeting Contract Obligation 
Requirements. 


Small Business — Performance Evaluation Summary. 
Customer Satisfaction. 


Cycle Time. 


Or 0. ON 


Procurement Management Review (PMR) reviews as they relate to 
Purchase Card. 


Per discussions from NAVSUP, the source of most of their metrics is the Field 
Contracting Offices and the Naval Inventory Control Points (Reverse Auctions, Contract 
Closeout, NMCI, Cycle Time, PMR reviews). Contract closeout is a mixture of MOCAS 
data and field input. Some of the information is gathered via PMRS (Competition, Small 
Business, and PBSC), and Customer Satisfaction is done via a NAVSUP web site that 


customers log onto and complete. 


Appendix A represents the current metrics FISC Norfolk utilizes in measuring 
contracting performance. FISC Norfolk has developed a balanced scorecard approach in 
determining the efficiency and effectiveness of the contracting operation. The scorecard 


has three key success factors (Effectiveness, Efficiency and Sustainment): 


1. Effectiveness Index: This consists of three weighted individual 
metrics: 
a. Number of Customers Profiled. 


b. Catalogues and IDTC’s on Portal/EMALL. 


c. Customer Satisfaction Index. 
2. Efficiency Index: This consists of four weighted individual 
metrics: 


a Number of Intranet Hits. 
b. Number of Customers on the Portal/EMALL. 


Electronic Transactions Index. 


a 9 


Number of Paperless Review Boards. 
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Sustainment Index: This consists of four weighted individual 
metrics: 


Series standards profiled. 


b. Percentages of IDPs Executed. 
Cy Training requirements resourced. 
ai; Training hours met. 


The scorecard then has a list of initiative metrics. Each initiative will have several 


initiatives associated with them. The scorecard will capture at least one metric for them. 


The standards for growth metrics are typically determined by the fiscal month of the year, 


e.g. in December we should have accomplished 25% of the goal; or by the initiatives plan 


for implementation. Minimum grades or Statistical Process Control sets the standards for 


process metrics as appropriate. “Red/Yellow/Green” indicators are assigned to individual 


metrics. The initiative metrics are: 


s 2 
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k. 
1. 


m. 
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Customer Satisfaction Survey. 

Portal Customer Market Share. 

Percentage of Performance based Contracts. 
Reverse Auctions. 

Self-Assessment Findings. 

Small Business. 

Overage Contract Closeout. 

SPS Upgrade Effectiveness. 

Percentage of Customer Requirements met via the Portal/EMALL. 
Percentage of FISC Market Sourced. 

NMCTI Order Fulfillment rates. 

Employee Satisfaction. 

Alignment. 

NAVSUP’s Metrics 


The first thing to recognize when analyzing both NAVSUP and FISC Norfolk’s 


metrics is what was said back in Chapter II, namely, “it is easier to develop lower level 


scorecards that link together if the metrics for the CEO have been defined first”. You can 


develop scorecards at many different levels of the organization. The key is that they link 


to higher-level metrics within the command or to a higher echelon command. NAVSUP, 
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as a Systems Command, must focus on strategic and in most cases ASN driven metrics. 
FISC Norfolk, a lower echelon is interested in a different set of metrics (combination of 
strategic and operating metrics). That being said, all lower level metrics or operating 
metrics should feed into the higher level. Unfortunately, NAVSUP’s metrics appear to 
be merely counting and “putting out the fire” type metrics. For instance, how many 
Performance Based Service Contracts have you awarded? How many overage closeouts 
have you performed? What is your Competition rate? What are your Small Business 
Statistics? How many reverse auctions have you conducted and what are the savings? 
How many PMRs were conducted and how many received an unsatisfactory rating? One 
can argue that none of these metrics measure the efficiency of a contracting office. Some 
may allude to the effectiveness of the contracting office (customer satisfaction, NMCI 
Obligation rates), but overall it would be difficult to determine, just based on these 
metrics, if your contracting function is efficient or effective. For example, a monthly 
counting of the number of reverse auctions and performance based service contracts, 
when both are meant to be contracting tools one should consider when determining the 
best approach to meeting a customer’s requirements, does not provide you with an insight 
as to whether these tools were used appropriately and achieved the goal of satisfying the 


customers. There are two mistakes here. 


First, establishing reverse auctions as a dashboard metric for the Chief of the 
Naval Supply Systems Command does not seem appropriate. This measure does not 
provide an accurate barometer as to whether the contracting function is operating 
“efficiently and effectively to deliver combat capability through logistics”. If used at all, 


reverse auctions should be a lower level metric. 


Second, using a reverse auction goal and measuring an office against that goal 
sends the wrong message. It may motivate contracting offices to fit a round hole into a 
square peg. Reverse auctions may be an excellent tool under the right circumstances. 
The key is to ensure that your contracting officers are trained and know the value of all 
the tools available to them. Performance Based Service Contracts (PBSCs), Small 
Business, and Contract Closeout fall into this same category, with a major difference, 
ASN has mandated these metrics. ASN requires that 50% of all eligible service contracts 


must be performance based by FY 05. Again, similar to reverse auction, one can argue 
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that a performance based service contract is just another tool for a contracting officer to 
consider when determining the best way to satisfy the customer. In addition, assigning a 
performance based services contract goal against a contracting office violates one of the 
tenets of selecting a metric; “the metric owner has the ability to influence the metric”. 
While contracting offices have a duty to train requirements generators in the benefits of 
performance-based contracts, they do not define the actual requirements. Many times 
the PCOs find themselves struggling to convert a non-performance based statement of 
work into a performance based statement of work while still satisfying the basic 
requirements of the service requested. There may be a strong argument to utilize a 
performance based contract, however, as GAO pointed out (GAO “Contract 
Management”: Guidance Needed for Using Performance Based Service Contracting), 
many still do not know all the steps required in defining a performance based service 
contract and more guidance is needed before determining the actual benefit of this type of 


contract. 


NAVSUP’s one metric that lends itself to efficiency is the Cycle Time Metric. 
However, this metric also has some problems. Cycle time measures the percentage of 
time you award a contract/order in less than 20 days for acquisitions less than or equal to 
$25,000 and 30 days for acquisitions between $25,000-$100,000. One of the problems 
associated with this metric is performance motivation. On many occasions, the customer 
does not require a contract/order within the 20 or 30-day goal (finding issues, required 
delivery dates, sending in a requirement 60 days before a contract is needed). On the 
other hand, some customers need their requirements filled immediately. PCOs must be 
capable of prioritizing workload without having to worry about meeting some arbitrary 
goal that does not provide a measure for efficiency and can seriously hinder your 
effectiveness. Most customers would not consider you too effective if you awarded their 
requirement in 19 days when they needed it in 2 days. This kind of metric may have a 
negative impact on customer relations and motivate poor business decisions. Not many 
managers would be happy to know that their PCOs would be willing to give up on the 
idea of negotiating for a better price because they needed to get an award done within the 
cycle time goals. In addition, NAVSUP did not consider the transaction costs associated 
with gathering and managing this metric. The Standardized Procurement System (SPS) 
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had to be modified and negotiators were required to track the reasons as to why they did 
not meet the cycle time goals and these had to be reported monthly. The only way to 
track the reasons is manually and is labor intensive. One has to question the value in 


gathering this metric when you analyze the cost of gathering it. 


All of NAVSUP’s metrics were established with no analysis of the management 
information systems available at each of their field offices. SPS does not have the 
capability to provide all of the metrics required. Also, all of NAVSUP’s metrics are 
“after the fact” or past metrics and does not look at Work-in-Process (WIP) or any future 
metrics. They utilize PMRS and MOCAS for some metrics and manual counts for others. 
One of the basic tenets of a good scorecard is to have a metric that looks as the past 
present and future in order to understand trends and motivate future performance. 

2; FISC Norfolk’s Metrics 

While it seems many of NAVSUP’s metrics fail to measure a contracting office’s 
efficiency and effectiveness, FISC Norfolk’s metrics, while lacking in several areas, 
makes a better attempt at measuring effectiveness and efficiency. FISC Norfolk has 
adopted the balanced scorecard approach as espoused by Mark Graham Brown. In their 
Metrics Manual (Appendix A), FISC Norfolk emphasizes that performance metrics 
should encourage performance improvement, effectiveness, efficiency, and appropriate 
levels of internal controls. Where practical, they should incorporate "best practices" 
related to the performance being measured and cost/risk/benefit analysis, where 


appropriate. The key elements of the performance metrics package should address the 


following: 
° Attaining Organizational Alignment. 
° Improvements in quality, cost reduction and/or avoidance. 
° Meeting customer requirements. 


° Meeting NAVSUP requirements. 
° Identification and development of resource requirements. 
The establishment of performance goals can best be specified when they are 


defined within three primary levels: 
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e Strategic Objectives: Broad, general areas of review. These generally 
reflect the highest goals based upon mission accomplishment and vision 
achievement. These metrics will comprise the corporate level metrics. 


e Operational Criteria: Specific areas of accomplishment that satisfy major 
divisions of responsibility within a function. These metrics are generally 
mission focused and form the underpinning of strategic metrics. 


e Waring Measures: Metrics designed to drive improvement and 
characterize progress or problems within each operation. These are 
specific quantifiable goals based on individual expected work outputs of 
critical processes or contributory functions. 


The SMART test is frequently used to provide a quick reference to determine the 


quality of a particular performance metric: 


S = Specific: clear and focused to avoid misinterpretation. Should include 


measure assumptions and definitions and be easily interpreted. 


M = Measurable: can be quantified and compared to other data. It should allow 
for meaningful statistical analysis. Avoid "yes/no" measures except in limited cases, such 


as start-up or systems-in-place situations. 
A = Attainable: achievable, reasonable, and credible under conditions expected. 
R = Realistic: fits into the organization's constraints and is cost-effective. 
T= Timely: doable within the time frame given. 


However, like NAVSUP, in many respects, FISC Norfolk has failed to adequately 
measure the effectiveness and efficiency of their operation. The detachments that make 
up the FISC Norfolk Acquisition Group (Philadelphia, Washington, Hampton Roads and 
Norfolk Naval Shipyard) do not share the same management information system. They 
are currently attempting to get all of their detachments under the Philadelphia Oracle 
based management information system. Failure to have a standardized system can create 
inequities and in some occasions require manual counts of data. Before a metric is 
established, one should ensure that all participants have the ability to simply measure it 
via a report out of a database. Unfortunately, some of the metrics established by FISC 
Norfolk does not look at efficiencies or effectiveness because some of the contracting 
offices do not have the ability to easily gather these metrics. The researcher is providing 


a model as part of his recommendations. The model proposed is assuming that all offices 
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have the same management information system. This is a required element of the model 


because it will ensure that all have the capability of measuring. 


Some of the problems with FISC Norfolk’s metrics are the complexity of the 
measures. Many individuals the researcher interviewed understood the importance of 
gathering a specific measure, but did not understand how they were being graded against 
that metric. For instance, on page 11 of Appendix A, FISC Norfolk provides a sample of 
the efficiency index. It was difficult to understand where the figures for Data and Score 
came from. Most understood the fact that you would have different weighting assigned 
to different measures as some are more important than others, but the overall metric’s 
scoring sheet was sometimes difficult to understand. Also, some metrics fall into the 
same category as NAVSUP’s in that they measured processes that FISC Norfolk had 
little to no influence over. Specifically, the Portal/EMALL metric assumes that we have 
the ability to add contractors and catalogs to the EMALL. While it is true that we may 
recommend and encourage contractors to have their catalogs added to the EMALL, we do 
not control the total process and many times it takes months in the DOD EMALL process 
chain to actually get a contractor/catalog added to the site. With regards to this metric, 
FISC Norfolk should concentrate only on the areas that they can influence (# of 
customers profiled) and disregard the measure of catalogs and Indefinite Delivery Type 


Contracts (IDTCs) added to the site. 


Another problem with the current metrics is the Customer Satisfaction measure. 
This measure is controlled by NAVSUP and requires the customers to access NAVSUP’s 
website and complete a customer satisfaction survey. Many customers have expressed 
dissatisfaction with this process mainly because they are being asked on a quarterly basis 
to fill out a satisfaction survey. In addition, there are many different customers 
associated with some Commands and it is often difficult to identify what specific 
customer within that Command needs to fill out the survey. Also, on some occasions, the 
sample size used to evaluate customer satisfaction was so small that the sample could not 


be relied on to provide an accurate measure of the effectiveness of customer support. 


There are some positives to the current FISC Norfolk balanced scorecard. For 


example, FISC Norfolk examines all aspects of the contracting function, including 
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employee and strategic measures. While the operational and customer satisfaction 
measures account for some aspects of efficiency and effectiveness, employee measures 
help the organization understand the overall health of the organization. By utilizing this 
balanced approach the organization can better understand where they need to adjust in 
order to better align themselves. They can focus on those specific areas that need 
additional management oversight. This has been most beneficial in the employee 
satisfaction measure. FISC Norfolk has been able to identify specific concerns of the 
employees through employee surveys, which has allowed them to focus their efforts in 
this area. Also, FISC Norfolk has developed a Quality Assurance Plan (QAP) that has 
enabled them to measure the quality of their operations. By doing quarterly self- 
assessments, the organization has been able to promulgate on an enterprise basis, lessons 
learned and more importantly, identify training needs of the workforce. By adopting this 
approach of self-assessment, FISC Norfolk has established a very robust internal training 
program and standardized many of their internal operations, which has led to a more 
efficient contracting function. FISC Norfolk has accomplished this by establishing an 
INTRANET, also known as the FISC I-Web, for the acquisition function. The FISC 
Norfolk acquisition group intranet is a secure, internal website designed for use by all 
acquisition personnel for convenient access to contracting and administrative information 
in a web based environment. The I-Web is an information resource that provides all 
necessary web sites/links and information to support the acquisition function in a 
centralized location. The I-Web provides quick access to and sharing of 
documents/policy updates and many acquisition issues that are important to the 
contracting community. By standardizing operations, all contracting personnel now 
utilize the same clearance templates, and have instant access to lessons learned. The I- 
Web also acts as a centralized area for communities of expertise exchanges, where 
negotiators can have instant access to expert advice. FISC Norfolk also utilizes the I- 
Web to post results of the balanced scorecard. This allows employees to see how the 
contracting function is performing and more importantly, it acts as a communication link 
between upper management and the working level. FISC Norfolk provides monthly 
metric briefing to their workforce, which promotes understanding and feedback. In 


Chapter IV, the researcher will propose a model (Appendix B) that addresses the 
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weakness of the current metric process to go along with some of the strengths of the 


current process. 
D. DEFENSE CONTRACT MANAGEMENT AGENCY (DCMA) METRICS 


As a member of a contracting office that handles requirements from their 
inception (Pre-Award) until their disposal (Post Award), the researcher was interested in 
discovering what metrics DCMA used in evaluating their post-award function. The 
following information was given to the researcher from the Philadelphia regional DCMA 
office. DCMA looks at nine metrics. The metrics are listed by title, definition, intent, 
computation and data source: 

1. Title: Prime Contracts On-hand 

Definition: The quantity of contracts assigned for primary 


administration at the end of the report period. 


Intent: This metric is used to track workload data for internal and 
external customers. 


Computation: The sum of all prime contracts on-hand as the 
contract administration office at the end of the report period. 


Data Source: Mechanization of Contract Administration Services 
(MOCAS). 


2. Title: Obligated Amount of Prime Contracts On-Hand 


Definition: The obligated dollar amount of prime contracts 
assigned for administration at the end of the period. 


Intent: This metric provides workload data for internal and external 
customers. 


Computation: The sum of the obligated dollar amounts of all 
prime contracts on-hand at the end. 


Data Source MOCAS. 
3. Title: Unliquidated Amount of Prime Contracts On-Hand 


Definition: The unliquidated dollar amount of open prime 
contracts assigned for administration at the end of the period. 


Intent: This metric is used to track workload data for internal and 
external customers. 


Computation: The sum of the unliquidated dollar amount of all 
prime contracts on-hand at the end of the report period. 


Data Source: MOCAS. 
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Title: Percent of Schedule On-Time 


Definition: The percent of line item schedules delivered before or 
during the month of their original delivery dates. The term “line 
item schedule” refers to a Contract Line Item (CLIN)/subCLIN 
with a single deliverable group, or in the case of a CLIN/subCLIN 
having multiple deliveries, each of the individual groupings 
created to account for differing destinations and/or delivery dates. 


Intent: On-time deliveries enable customers to maintain readiness 
requirements. 


Computation: Divide the quantity of line item schedules due 
during the report month that were delivered during or before the 
report month by the total quantity of line item schedules due 
during the report month. Multiply the result by 100. Use the 
original delivery date of the contract if the contract was not 
modified due to a Government caused delay OR the most recent 
modified delivery date due to a Government caused delay. In 
MOCAS this is the “Delivery Schedule Date” field. 


Data Source: MOCAS and an Internal Management Information 
System. 


Title: Outstanding Delinquent Line Item Schedule Reduction 


Definition: The number of undelivered line item schedules whose 
current delivery dates have past and whose entire required quantity 
has not been shipped. 


Intent: An unacceptable large number of outstanding schedules 
reside in our database systems. Many of these are data 
misrepresentations. Others are legitimate delinquencies. 
Erroneous data represents true contractor performance to our 
buying activity customers and is an unreliable input to contractor 
risk assessments and subsequent surveillance activity. Legitimate 
delinquencies hinder the readiness requirements of the Armed 
Services and require top-level visibility. 


Computation: Count the line item schedules in which the 
scheduled quantity exceeds the ship quantity, and the month of its 
current delivery schedule is past. 


Data Source: MOCAS. 


Title: | Undefinitized Contract Actions (UCAs) On-Time 
Definitized UCAs and Overage UCAs 


Definition: UCAs not definitized within 180 days for the date 
issued are overage. 


Intent: To reduce the overall number of overage UCAs. 
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Computation: The overage undefinitized contract action rate is 
calculated by dividing the quantity of overage UCAs by the 
quantity of active UCAs during the reporting period. 


Data Source: Pricing and Negotiation AMS module. 
Title: Contracts Closed On-Time 


Definitions: Achieve closeout of contracts within the FAR 
mandated timeframes. 


Intent: To close al contracts 87% of the time within mandated 
timeframes and to measure our contract closure similar to how our 
customers measure our contract closeout success. Also, to allow 
for cleanup of MOCAS Section 2 without being penalized by the 
old measurement of overage percentage. 


Computation: The percentage of time all contracts close within 
their FAR mandated timeframes. 


Data Source: MOCAS Part A and B, Section 8. 
Title: Contracts Past Final Delivery Date 


Definition: Monitor percentage of contracts past their final 
delivery date (FDD). 


Intent: Maintain contracts more than 180 days past their final 
delivery date at 5% or less as the end of the month and to 
accurately identify all physically completed contracts for our 
customers and ensure that these contracts are in MOCAS Section 
2 


Computation: Percentage of Contracts in MOCAS part A, Section 
1 that are more than 180 days past their FDD. 


Data Source: MOCAS Part A, Section 1. 


Title: Quantity of Overage Contracts for Closure 
Definition: Reduce the quantity of overage contracts. 


Intent: Reduce the quantity of overage contracts by at least 50%. 
In addition the intent of this metric is to determine that the quantity 
of overage contracts does not disproportionately increase in the 
efforts to meet the current performance target and also allow for 
cleanup of MOCAS Sections without being penalized by the old 
measurement of overage percentage. 


Computation: The quantity of overage is computed by 
determining the quantity of contracts in the population that are 
overage for closure. 


Data Source: MOCAS Part A, Section 2 overages. 
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The researcher considers all of these metrics to be counting metrics and provide 
little to no insight in organizational effectiveness and efficiency. It seems that DCMA 
has a good handle on how much work they have, but no idea if they are performing 
efficiently. The nine metrics listed above look more like a wish list of strategic goals 
(e.g. reduce the quantity of overage contracts). The problem with these types of metrics 
is that you may never know if you are doing an efficient job or not. For example, you 
may be doing an outstanding job in closing out contracts, but the number of overage 
contracts may increase (due to current contracts becoming overage) leading you to 
believe that you are not being efficient. They do not address the overall process. A 
better measure may be to focus on a priority system for closing out contracts, with an 
initial focus on closing out contracts whose funding expires that fiscal year, with least 
important priority being the simple fixed price instruments you can easily close out. By 
focusing on the numbers of closeouts you have accomplished in a specific month, you are 
motivating employees to focus on the “low hanging fruit” and not the more critical 
closeouts where you may actually be able to recoup funds and not have to use current 
year funding to pay for past year performance. DCMA’s metrics seem to mirror the top 
ten measurement mistakes that Mark Graham Brown points out in his book “Winning 
Score”. They provide no real insight into the effectiveness or efficiency of the contract 


administration function. 


What should DCMA consider when measuring their processes? DCMA should 
consider the same things that FISC Norfolk and NAVSUP need to consider. Mission 


related metrics should meet the following characteristics (Graham Brown, “Winning 


Score”, 2000, p. 48): 


1. There are no more than 20 metrics. 
2 Metrics are important and link back to overall goals and mission 
statements. 


Metrics are a roughly equal mix of past, present and future. 


4. Counting measures are based upon meaningful things, where possible. 
5. Judgment metrics are based upon ratings established using specific 
criteria. 
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6 Metrics are as objective as possible. 

he Metrics will drive the right behaviors from employees and partners. 

8 Collecting data on the metric is feasible and cost effective. 

9 Employees can’t easily cheat on measures. 

10. Most employees can understand measures. 

11. | Measures lend themselves to be tracked on a regular basis so that changes 
can be detected in performance when there is still time to do something 
about it. 

12. Measures depict aspects of performance over which we have quite a bit of 
influence. 


These recommended tips should be examined when metrics are established. It provides 
the metrics manager a checklist to ensure the proposed metrics are adequate and meet the 
needs of the organization as well as help determine which metrics need to be added or 
deleted. 

E. CHAPTER SUMMARY 


This chapter has provided the results and analysis of the data. The results from 
the research effort revealed affirmation of the literature review. Agencies are still 
struggling to establish meaningful metrics and are having difficulty in meeting the 
requirements of the Government Performance Results Act. The research also indicated 
that the Navy has not ever measured the efficiency of a contracting office (to the best of 
the researcher’s knowledge) and believes based on the results of their studies, that all 
contracting personnel are alike and have the same skill sets required to satisfy all types of 


requirements. 


Next, the chapter examined the results of a study performed in the private sector 
that focused on CEOs and CPOs. The results showed some significant dichotomies 
between what a Chief Purchasing Office considers important to the effectiveness of a 
purchasing office, compared to what a Chief Executive Officer finds relevant. While 
CEOs were more interested in the quality of the supplies and the quality of the 
relationships purchasing personnel establish with both suppliers and internal customers. 
CPOs, were interested in the savings they accrue from price negotiations. This contrast 
can many times lead to opposite results. By driving down the price for a product or 
service, the CPO may be reducing the quality received. 
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The chapter then focused on the current NAVSUP and FISC Norfolk metrics. 
The research pointed out many limitations in the current metrics. The research also 


showed some positive aspects of FISC Norfolk’s current process. 


The DCMA metrics were then examined to gain a better understanding of the 
post-award measures that the administrative contacting community employs. The 
research found some problems with DCMA’s current metrics. Many of DCMA’s 
measures do not address efficiency and effectiveness. Instead, they only “count” how 


many things they are doing. 


Finally, the results obtained during this research effort highlight the fact that 
measuring efficiency and effectiveness of a contracting office is an extremely difficult 
endeavor. Mixing subjective measures with objective measures is difficult and 
sometimes unfeasible. Balancing the needs and desires of stakeholders make it extremely 
difficult to achieve organizational efficiency and effectiveness. A comprehensive 
instrument or model needs to be developed to help contracting offices measure how well 


they are performing. 


63 


THIS PAGE INTENTIONALLY LEFT BLANK 


64 


IV. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


This research effort provides an examination into the contracting function with a 
specific emphasis on efficiency and effectiveness. The research effort used a qualitative 
approach to gathering and analyzing data from a Navy and private industry perspective. 
While the analysis of the Navy Studies were based on summaries provided by a private 
contractor, it still provided enough information to analyze what and why the Navy has 
invested significant resources in studying the contracting function. The research also 
showed the limitations of these studies. Based on the research, the following conclusions 
are supported by the data gathered: 


1. The Navy Has Not Performed An Efficiency or Effectiveness 
Evaluation of a Field Contracting Office 


The research indicates that the Navy studies did not limit themselves to a field 
contracting office, in fact, most of the studies were performed to determine the most 
effective way to organize the contracting function in the Systems Commands or the Navy 
overall. The research also indicates that most of the studies draw on anecdotal evidence 
and therefore, they only generally address and hypothesize on the efficiency and 
effectiveness of a contracting office. The studies provide a background into what the 
Navy leadership considers relevant when assessing the effectiveness of its contracting 
function but they did not examine a contracting office as an independent entity, nor did 
they attempt to measure the productivity of a contracting office. Finally, the research 
indicates there still seems to remain a perception that all contracting professional (also 
known as “1102s”) are created equally with the same skill mix to acquire any type of 
supply or service. 


2. Purchasing Managers Do Not Always Understand How They Are 
Being Evaluated 


An analysis of the private sector revealed that CEOs and CPOs have different 
opinions when it comes to determining the efficiency and effectiveness of a purchasing 
office. This research showed the private industry struggles as much as the public sector 


in determining what measures should be used in determining the efficiency and 
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effectiveness of a purchasing function. The research showed that CEOs are mainly 
interested in measuring the effectiveness of a purchasing office, with less emphasis on 
efficiency. They tend to leave that up to the Purchasing Manager. Specifically, CEOs 
were interested in the quality of the supplier’s product as well as the relationship fostered 
with the supplier. In the meantime, the research indicated that CPOs were interested in 
saving money and getting better deals for the organization and did not focus on the 


quality of the supplier. 


On the public side, the Navy and DCMA share the same problems that the private 
sector face. The measures that are currently employed do little in determining the 
effectiveness and efficiency of a contracting office. Many of the current metrics motivate 
inefficient performance and leaves mangers of field contracting offices struggling to 
understand what they should be doing. 


2: Efficiency and Effectiveness are Difficult to Measure at the Systems 
Command and the Field Contracting Office Level 


The research demonstrated that Systems Commands and Field Offices have 
different perspectives when it comes to measuring efficiency and effectiveness. The 
research indicates that the current metrics required by NAVSUP does not measure the 
effectiveness and efficiency of a contracting office and fails in many respects to meet the 
requirements of the GPRA. The research also showed that many of the counting” 
metrics currently deployed come from higher levels within the Government (ASN and 
SBA) and they have different motives, mostly political, for their actions. The research 
inferred that many times qualitative metrics like efficiency and effectiveness should only 
be captured at the operational level and the higher-level metrics should focus only on 
mission and strategy. The research indicates that the Navy has a difficult time linking 
their metrics to their overall strategy. 


4. The Navy Does Not Have a Model to Measure the Effectiveness and 
Efficiency of a Contracting Office 


Based on the results of the research conducted concerning the definitions of 
efficiency and effectiveness, the Navy’s current measurement process does not 
adequately measure its contracting offices. To the best of the researchers knowledge, 


there is no indication that the Navy has a model that evaluates all aspects of the 
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contracting office. The research shows many holes in the current metrics throughout the 
hierarchy. Additionally, there is no enterprise-wide metrics employed by the Navy to 
measure contracting offices against. This void creates considerable problems when 
measuring the contracting function. The research of the many studies conducted by the 
Navy indicates that the hierarchy may not be interested in how efficient a contracting 
office performs. Instead it appears that they are more interested in appeasing the interests 
of their many stakeholders. 


B. RECOMMENDATIONS 


Based on the results and analysis of the data reviewed, several recommendations 
for selecting the proper metrics were presented in Chapter II]. These recommendations 
should be followed when determining what metrics a contracting office should employ to 
measure its effectiveness and efficiency. The researcher is proposing a model that is 
based on information obtained at FISC Norfolk Detachment Philadelphia. The model is 
based on a balanced scorecard approach that divides the organizational metrics into 4 
different areas: Strategic, Operational, Customer and Employee. All four areas are equal 
in importance and the metrics provided in each area must be continually updated to 
ensure success. The Strategic area will consist of several kinds of metrics that address the 
overall output of the organization (obligation and action rate) as well as strategic mission 
metrics (how do you support the war fighter) and vision metrics (actions and obligations 
per Federal Supply/Service Code (FSC) and other strategic type metrics. The operational 
area will look at a current breakdown of work-in-process by type of action and will depict 
a manage-to-milestones approach that will be explained later. The customer metrics will 
focus on the type of support we provide for the customer including a spend analysis of 
their requirements and any new initiatives we are providing for them. Since NAVSUP 
gathers the customer satisfaction metric for its field contracting offices, this metric will 
not be included in the field model. The employee metrics will focus on available training 
(courses and dollars), intern program management and significant employee issues and 


assignments. 


A basic requirement for using this model is that you must have a management 
information system that allows you to extract the proper metrics. FISC Norfolk 


Detachment Philadelphia uses an Oracle based management information system called 
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ORACIMS. ORACIMS allows one to measure processes to significant milestones, 
which is important to a field activity like FISC Norfolk Detachment Philadelphia. A 
brief description of the detachment is provided so one can better understand the metrics 
model that is proposed as Appendix (C). FISC Norfolk Detachment Philadelphia is a 
Navy field contracting office that provides contracting services to over 175 customers. A 
significant majority of the actions done by this detachment are in the services area and 
are also high dollar type requirements. The detachment annually obligates approximately 
one billion dollars. Approximately 2% or 20 million is obligated in the Simplified 
Acquisition Threshold ($0-$100,000) area. Because Philadelphia has such a significant 
amount of Large Procurements (over $100,000), their internal management information 
system is based on a management to milestones philosophy that divides all new 


procurement work into four areas: 


1 The Unsolicited Phase. 

2 Solicitation Phase. 

3: Technical evaluation Phase. 

4 Negotiation Phase. 

The management information systems also track all requirements form “cradle to 
grave” and has the capability to track many types of administrative actions as well as 
unmeasured work (the system was recently updated so buyers could keep track of the 
amount of time they spend on acquisition planning, answering administrative issues, 
customer meetings). This has become important recently because of the Navy 
“Transformation” effort underway. It provides NAVSUP and ASN an idea of the amount 
of time dedicated to unmeasured workload. The researcher believes that the model being 
presented in Appendix B will go a long way in measuring the efficiency and effectiveness 


of a field contracting office. 


The following provides an explanation of the metrics displayed in Appendix B: 


1. Strategic Metrics 
- Actions and Obligations: The first 2 slides compare actions and 
obligations over the past 2 fiscal years. This gives managers a picture of 


the current year obligation rates and gives them trend data. The data 
source for this metric is PMRS. 
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Large Purchase Top Ten Suppliers by Dollars: Managers can use this 
type of information for leveraging with suppliers on future requirements. 
The data source for this metric is PMRS. 


Actions and Obligations for Services/Supplies: Provides managers an 
indication of the type of work you are performing. There is a significant 
difference between service contracting and supply contracting. Much 
more oversight is required in a services contract and for the most part our 
office retains contract administration over the service contracts we issue. 
It is important that managers understand the significance of this metric. 
The data source for this metric is PMRS. 


Large Purchase Top Ten FSCs by Actions: This metric gives managers 
an indication of the types of services they are buying. A corresponding 
metric for supplies would be appropriate if your office procured a majority 
of them. While additional exploration of the exact type services is 
required, it still provides management with a good idea of the type of 
services the office is familiar with and maybe even expertise at acquiring. 
This type of metric fits into the strategic enterprise area as it may provide 
headquarters with insight into where they may want to centralize certain 
types of services or supplies as was recommended in several of the studies 
analyzed in Chapter HII. The data source for this metric is PMRS. 


Competition: This metric is included in the Strategic area because it is 
required by NAVSUP. A month-to-month compilation is recommended to 
provide trend data. While NAVSUP already gathers this metric for their 
dashboard, it is still important to keep it at the field level, so you are aware 
of significant changes that may affect your goal. The data source for this 
is PMRS. A recommended change to this metric is to calculate the 
number of dollars that have been successfully broken out of the sole 
source area. This metric can be gathered via the ORACIMS database as 
long as the negotiators are diligent in recording this type of transaction. 
The researcher would also recommend some recognition to those 
individuals who breakout historically sole source actions. 


E-Business: This metric is also included in the Strategic area because it is 
required by NAVSUP. OSD and ASN have a significant interest in 
making contracting paperless. OSD had a goal to make all contracting 
paperless by CY 2001. Their assumption was that SPS would be fully 
implemented by then and all requirements both received and completed 
would be done paperless. Unfortunately, DOD has never agreed on an 
electronic requirements generating system and not all awards are done in 
SPS for various reasons (urgency, NMCI orders are exempt from SPS 
because they have their own electronic systems). We include this metric 
because it allows us to focus on our top requirements providers and jointly 
try to determine a way to receive electronic purchase requests. The 
problem with this metric is the data sources. Some information is pulled 
from SPS, while the rest is pulled from ORACIMS (we added a block to 
our ORACIMS database to get this information). If you do not have a 
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management information system, you will have to manually gather this 
data. 


Anticipated Future Obligations: Forecasting future obligations and 
business is important for planning and resourcing, especially if it is an 
indefinite delivery type contract where we anticipate issuing significant 
orders. This metric is intended to look at your top ten future obligations. 
The data source for this metric is ORACIMS. 


Contract Closeout: This metric is included in the Strategic area because 
it is required by NAVSUP and ASN. The Under Secretary of Defense has 
a significant interest in reducing the amount of open contracts. The 
contract closeout status metric just counts the amount of closeouts 
accomplished. The data source for this is ORACIMS. The dollars 
deobligated gives management an idea of the amount of money returned to 
the customer. This also helps focus the employees on a specific area in 
close-out that has management attention and hopefully will motivate them 
to closeout those contracts where there is a possibility for recoupment 
before they focus on the “low hanging fruit”. The data source for this 
metric is ORACIMS and PMRS. The last closeout metric is required 
specifically by NAVSUP and must be updated monthly. The data source 
for this metric is MOCAS and ORACIMS. 


Small Business Goals: Again, this metric is included in the Strategic area 
because it is required by NAVSUP and ASN. It provides management 
with a monthly snapshot of how they are faring against their assigned 
goal. The data source for this metric is PMRS. 


Business Development and Major Program Support: While these two 
metrics may not fit into a measurement of efficiency and effectiveness, 
they do provide management with a monthly picture of new developments 
and a running ledger of major programs supported. The metrics are 
beneficial when they are communicated to the employees so everybody is 
familiar with the type of programs the office supports. The data source for 
this metric is manual. 


In summary, the purpose of the strategic metrics is to give management insight into the 


various strategic interest areas of NAVSUP and ASN, while also focusing on the field 


office. 


Operational Metrics 


Cycle time metrics: This metric could have possibly been placed in the 
strategic area because is required by NAVSUP, but it fits better into the 
operational area. Again, this metric has the possibility of promoting poor 
business decisions. However, used properly, it can give you insight into 
possible systemic issues if you are consistently missing your goal. The 
key to this metric is understanding why the goal is missed and identifies 


70 


the root causes for missing the goal. The data source for this metric is 
SPS. 


Competitive Sourcing and Streamlined A-76 Process. This metric 
could also fit in the strategic area because of the high visibility associated 
with this program. The researcher recommends it be placed in the 
operational area because of the effect the A-76 process has on 
Procurement Administrative Lead Time (PALT), which is an efficiency 
type metric. Comparing the time it takes to da full study compared to a 
streamlined is important to understand and communicate to higher levels. 
It is also an important metric because a Government win results in no 
obligations in the DD 350 system. The A-76 process is sometimes a long 
resource intensive process that receives no “credit” in the PMRS database. 
Therefore, it is critical to have a total understanding of the workload in 
this area. The data source for these metrics is ORACIMS. 


Work-in-Process Profile, Acquisition Strategy and Contract Type: 
These metrics provide management with an instant picture of what 
encompasses their current workload. By breaking _ the 
folders/requirements out by new procurement, Type-I Delivery Orders 
(fixed priced orders under single of multiple award contracts), Type II 
delivery orders (Time and Material, Labor Hour, Cost Type orders under 
single or multiple award contracts) and other provides a quick picture of 
the type of work in process. The acquisition strategy metric also provides 
you with an instant picture of the strategy the negotiator will be employing 
in meeting the requirements. Tracking by contract type can also provide 
management with important information for allocating resources. The 
large amount of cost type contracts may require moving resources around 
the command or moving work depending on the overall strategy of the 
detachment. As stated above, usually more oversight is required in service 
contracting and especially, cost type service contracts. The data source for 
these metrics is ORACIMS. 


New Procurement Summary: This is where the management to 
milestones philosophy comes into play. In an organization that has a 
significant amount of large dollar complex contracts, it is important to 
install some kind of milestone system. The purpose of the milestone 
system is to ensure that we are attempting to operate as efficiently as 
possible. The Philadelphia Detachment has divided the miles tones into 
four specific phases: The Unsolicited Phase, Solicitation Phase, Technical 
Evaluation Phase, and the Negotiation Phase. In addition, the new folders 
are further delineated by Competitive, Sole Source, Competitive 
Commercial Items and Sole Source Competitive Items. The reason for 
this separation is the different rules that apply to commercial items (FAR 
Part 13. 5 and other FAR Part 12 rules). This is just an overall picture of 
the new work-in-process. The next four metrics provide a more in-depth 
look at the work-in-process. The data source for this metric is ORACIMS. 
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Unsolicited Phase: Management has established a baseline that all new 
procurements should be solicited within 45 days (30 days for commercial 
items) after receipt. This standard represents the average time it takes to 
get to the solicitation phase and management understands that with good 
reason this standard may not be met. Management should review the 
standard at least quarterly and adjust as appropriate. The detachment uses 
a red-yellow-green rating to each new folder in this milestone area. The 
purpose is to encourage the workforce to solicit their requirements as 
quickly as they possibly can without sacrificing quality. A red rating may 
not be a bad thing if proper justification accompanies it. This metric may 
also allow managers to identify some systematic issues concerning the 
unsolicited phase and may lead to fixing root cause problems. The data 
source for this metric is ORACIMS. 


Solicitation Phase: The goal of the detachment is to have solicitations 
close in 45 days or less (30 days or less for commercial items). Similar to 
the unsolicited phase, management understands that there may be 
instances that this can’t be met. The purpose of this metric is to gain 
insight into the reasons solicitations are delayed and resolve any root 
cause problems. The data source for this metric is ORACIMS. 


Technical Evaluation Phase: The goal of the detachment is to have 
technical evaluations completed in 45 days or less. While in many 
instances the negotiator can’t control the time it takes for a technical 
person to perform his duty, it is important for management to understand 
what causes the delay. The detachment experienced a benefit of tracking 
this metric. They found the root cause of late technical evaluations (on- 
site vice off-site evaluations). The data source for this metric is 
ORACIMS. 


Negotiation Phase: The goal of the detachment is to conclude 
negotiations within 45 days after the solicitation closes. This metric 
serves in the same capacity as the other three mentioned above. The data 
source for this metric is ORACIMS. 


Delivery Orders: The quantity of delivery orders being written today are 
increasing at an alarming rate and has become a significant area of interest 
at FISC Norfolk Detachment Philadelphia. What’s also growing at an 
alarming rate is the number of multiple award contracts. This metric is 
broken down by order type (I or II) because fixed priced delivery orders 
(Type I) are less complex than cost type orders (Type II) and further 
divided by competitive and non-competitive. This is an important feature 
of the delivery order process as some competitive delivery orders can 
require technical evaluation factors as well as cost and prices and literally 
result in a mini-competitive best value source selection. The standard for 
non-competitive Type I orders is 15 days or less and 45 days or less for 
Type II orders. The goal of competitive orders is 45 days or less no matter 
what type of order. There was originally one standard in this metric, 
however, over time management noticed the trend associated with 
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multiple award type orders and decided to separate them out so they have 
a better picture of the delivery order process. The data source for this 
metric is ORACIMS. 


Other Procurement Actions: This provides management with the rest of 
the workload picture. A renewed focus on contract management has been 
undertaken by the detachment in order to gain a better understanding of 
the amount of time and resources expended in contract administration. 
Not all modifications are equal and the goal of this metric is to separate 
out the simple ones (exercising an option and funding action) from the 
more complex (Change orders). Simplified Acquisition Purchase is also 
captured (most of these are simple). This provides management a 
snapshot of part of the Contract administration picture. The source data 
for this metric is ORACIMS. 


Large Purchase and Simplified Acquisition Self-Assessment Findings 
and Self-Assessment Recommendations: It is great to have a picture of 
the entire workload. The more important question is how well are we 
doing? What is the quality of the products we produce? This is where the 
quality assurance management program comes in. The idea is to conduct 
quarterly self-assessments on at least 10% of the awarded population to 
gain a better understanding of the quality of the contracting operation. 
These metrics help gauge how efficient and effective an organization is 
performing. The first step in the process is to establish a quality assurance 
plan that identifies the key acquisition processes in your command. Then, 
after you define the key acquisition processes, you assign a standard that 
you expect the organization to achieve in that process. For FISC Norfolk 
a team determined the standards as acceptable error rates. For example, it 
is expected that 90% of all files sampled will have an adequate Business 
Clearance. After conducting the self-assessments it is critical to post the 
results and recommend specific actions whether it be a change in business 
processes or additional training. This metric is currently gathered 
manually. However, an automated version is currently being studied. 


In summary, the operational metrics are focused on the efficiency of the 


contacting function. The model suggests a measuring to milestones approach for Work- 


in-process. Most of the operational metrics have a distinct focus on the present. It leaves 


the past metrics to the strategic and customer areas and concentrates on the current 


operations. 


3. 


Customer Metrics. 


Large Purchase Top Ten Customers by Actions and Dollars: These 
metrics focus on the top ten customers either by action or by dollars. This 
gives management some insight into what customer and how much we are 
supporting them. This does not rank customer by importance. It provides 
management with an idea of the resources that may accompany the work 
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of each customer. It by no means tells the entire picture. Some customers 
have small quantities of actions, but may be critical to the operations of 
the war fighter. The purpose of this metric is to give management a rough 
order of magnitude. The data source for these metrics is PMRS. 


- What do we buy for our top ten customers: These metrics focus 
specifically on what we acquire for our customers. These metrics should 
be utilized in monthly meetings with the customer to determine possible 
economies by combining requirements and also provide the customer with 
a little portfolio of where they are obligating their money. These metrics 
provide a beginning framework in establishing a Customer Relationship 
Management (CRM) Program. In addition, these metrics could be used on 
an enterprise-wide basis not just for NAVSUP but also for the entire 
Navy. The data source for these metrics in PMRS. 


In summary, one metric that seems missing in the customer area is customer satisfaction. 
Historically, the field offices use to survey customers quarterly or semi-annually. In 
1999, NAVSUP decided that they would take over the customer satisfaction survey on an 
enterprise-wide basis. NAVSUP does these surveys quarterly and provides the results to 
the individual field offices. The field offices have found some flaws to this process, but 
do not wish to burden customers with more surveys to complete. While it is not included 
in any of the proposed metrics, the researcher recommends monthly meetings (any 
medium) as a minimum to ensure that customer expectations are being met. The 
researcher was not able to develop a meaningful metric for customer meetings other than 
to keep track of them and report out any significant issues. 
4. Employee Metrics 
- Staffing: Although this is a “counting” metric, it is important to 
understand the turnover rate of your organization. The researcher 
recommends a four-year period. Any significant spikes in the chart can 
alert management to further investigate the root causes of departure. This 
metric may provide little information for a stable workforce. However, 
when you combine that with the training budget allocated to the 
workforce, it can give you a better idea of how much training dollars are 


available per employee. The data source for this metric is the human 
resource personnel database. 


- Training Statistics: This metric provides management with a monthly 
snapshot of how much training is being received and accounted for by the 
workforce. By using this metric, one can determine if the employees are 
accounting properly for the amount of training they are receiving. The 
data source for this metric is manual (sign-in sheets). 
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Training Budget for 1102s, 1105s and non-1102s/1105s. This metric 
provides management with a picture of the training budget and how well 
the organization is implementing its external training program. This 
metric does not include in-house training as that is considered a no-cost 
training program. While FISC Norfolk recommends an Individual 
Development Plan (IDP) execution rate, the researcher does not support 
that type of metric. Based on prior experience, the researcher feels that 
the IDP process has significant flaws. In the past few if any employees 
actually received the training they requested in their IDPs. Many of the 
reasons for not executing IDPs were budget driven (training dollars are 
continually dwindling) and most employees do not consider the IDP 
process seriously. They are tired of asking for specific training that is 
endorsed by their immediate supervisor, only to find that there is no 
funding available when the training is being offered. By knowing what 
the training budget is in advance of the fiscal year, managers will e able to 
allocate training dollars per employee. This will help employees select 
training courses within the budget allocated to them. This metric will help 
manage the training budget process. The data source for this metric id 
One-touch Financial (OTF). 


Next Generation Workforce: This metric keeps track of all of the 
training programs and employee projects and possible employee issues. It 
provides management with an idea of how robust the training program as 
well as employee involvement. Included in this metric should be any 
employee issues, other than confidential. The data source for this metric is 
manual. 


In summary, this employee area addresses the past, present and future. Not included in 


these metrics, but considered important are employee surveys. Employee surveys can 


often provide insight to the working conditions of the office. However, caution should be 


used when doing employee surveys. One of the major flaws associated with employee 


surveys, as experienced by the researcher, is the frequency in which they occur. Monthly 


or quarterly surveys are not recommended. For best results, the researcher recommends 


semi-annual or annual employee surveys. The most important aspect of any survey is 


that you perform some kind of action in response to the survey. To do nothing is 


unacceptable and taints the importance of the survey as a measurement tool. 


ANSWERS TO RESEARCH QUESTIONS 


Primary Research Question 


a. What Model Is Needed to Measure the Efficiency and 
Effectiveness of a Navy Field Contracting Office? 


(= 


The researcher has developed a model (Appendix B) for measuring the 
efficiency and effectiveness of a Navy Filed Contracting Office. The model is based on a 
balanced scorecard approach that divides the organizational metrics into 4 different areas: 
Strategic, Operational, Customer and Employee. All four areas are equal in importance and 
the metrics provided in each area must be continually updated to ensure success. 
Management should be analyzing the metrics on a monthly basis and correct any 
misalignment of the organization. This will also provide management with insights into 
where they need to concentrate their resources. For example, results of an employee survey 
that show employees do not understand where they fit in the organizational strategy or felt 
that they were lacking the adequate training to meet the mission goals, would allow managers 
to concentrate on communicating with employees and map out a training program that would 
satisfy their needs. The model also measures the areas that are required form NAVSUP and 
ASN, while also providing the field office with there own set of measures. 

2. Secondary Research Questions 


a. What is the Difference Between Efficient and Effective and Why 
Should We Strive for Both? 


As was discussed in Chapter IJ, efficiency refers to the capacity to produce 
results with the minimum expenditure of time, money, or materials. Efficiency thus 
focuses on the input-output ratio. To be efficient is to do things well, to attend to the 
internal organization by refining, formalizing, and elaborating on existing knowledge and 
making short-term improvements. Effectiveness on the other hand, is defined as 
“productive of results”. The focus is on doing the right thing and that is determined by 
outcome attainment. It comes from an understanding and interpretation of the exterior 
environment and what outcomes are required (e.g., Customer satisfaction, goal 


achievement, negotiated savings). 


Both efficiency and effectiveness play an important role in organizational 
performance, yet in the competition for resources, each can interfere with the other. 
Efficiency depends on focus, precision, repetition, analysis, discipline and control. 
Effectiveness, on the other hand, relies on experimentation, novelty and loosened control. 


While effectiveness thrives on experimentation, efficiency attempts to drive out any 


76 


uncertainties. Consequently, public managers need to make choices between the level of 


efficiency and effectiveness they intend to pursue. 


Efficiency looks at internal operations, while effectiveness is viewed as 
how well are we performing by the external environment (stakeholders). An organization 
can be considered quite effective by a customer, yet be significantly inefficient when you 


examine their internal operating procedures. 


So why do we need to strive for both? Simply stated, we are required to. 
As stated previously in this chapter “NAVSUP has embarked on a journey to better 
structure and align the organization to most effectively and efficiently deliver combat 
capability through logistics”. In addition to Navy transformation, the goal of the GPRA 
is to measure us by our outcomes and demonstrate to the taxpayer that we are an efficient 
and effective organization. It is also inherent in managing any type of business. If you 
want to keep a vibrant and successful organization, you must be efficient and effective. 


b. Where Do Most Field Contracting Offices Fit in the 
Organizational Configuration Model? 


HIGH 
FCO 





EFFECTIVENESS 
RESPONSIVE 
LOW 
Low HIGH 
EFFICIENCY 
Figure 4. Field Contracting Officer (FCO) Organizational Configuration 
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The researcher took an informal survey as well as examined several field contracting 
offices mission statement and strategic plans to determine where they fit on the 
organizational configuration. As depicted in Figure 4, most see themselves as a hybrid 
organization. Continuously striving for customer satisfaction (effectiveness) and supporting 
the war-fighter, while offsetting that with ways to reduce infrastructure and cycle time 
(efficiency). Most Field Contracting Offices find themselves being graded on how effective 
they are in meeting both customer needs and the multiple stakeholder interests. When asked 
how they balance efficiency with effectiveness most mentioned the balanced scorecard as an 
approach to measuring not just effectiveness and efficiency, but also the overall performance 
of the entire organization in how it meets its mission and supports its employees. 


Cc. What Metrics Are Typically Used to Measure Field Contracting 
Office Efficiency and Effectiveness and What Are the Issues 
and/or Limitations Associated with These Metrics? 

The following is a list of the current Contracting metrics that NAVSUP is 
gathering and presenting to the Chief of the Navy Supply Corps on a monthly basis as 
part of the Chief’s overall “dashboard” (see the Balanced Scorecard presented in Chapter 
Il): 


Number of Reverse Auctions conducted and saving achieved. 
Percentage of Performance Based Service Contracts (PBSC). 
Percentage of work Competed. 

Contract Closeout and the number of over-aged contracts. 


Navy and Marine Corps (NMCI) — Meeting Contract Obligation 
Requirements. 


Sot Fe es LEO 


Small Business — Performance Evaluation Summary. 
Customer Satisfaction. 


Cycle Time. 


SOLIS, ah AON 


Procurement Management Review (PMR) reviews as they relate to 
Purchase Card. 


FISC Norfolk’s metrics are located in Appendix A. As was stated in Chapter III, 
there are significant limitations to the current metrics. In Chapter II, we found “‘it is 


easier to develop lower level scorecards that link together if the metrics for the CEO have 
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been defined first”. You can develop scorecards at many different levels of the 
organization. The key is that they link to higher-level metrics within the command or to 
a higher echelon command. NAVSUP, as a Systems Command, must focus on strategic 
and in most cases ASN driven metrics. FISC Norfolk, a lower echelon is interested in a 
different set of metrics (combination of strategic and operating metrics). That being said, 
all lower level metrics or operating metrics should feed into the higher level. 
Unfortunately, NAVSUP’s metrics appear to be merely counting and “putting out the 
fire” type metrics. For instance, how many Performance Based Service Contracts have 
you awarded? How many overage closeouts have you performed? What is your 
Competition rate? What are your Small Business Statistics? How many reverse auctions 
have you conducted and what are the savings? How many PMRs were conducted and 
how many received an unsatisfactory rating? One can argue that none of these metrics 
measure the efficiency of a contracting office. Some may allude to the effectiveness of 
the contracting office (customer satisfaction, NMCI Obligation rates), but overall it 
would be difficult to determine, just based on these metrics, if your contracting function 
is efficient or effective. For example, a monthly counting of the number of reverse 
auctions and performance based service contracts, when both are meant to be contracting 
tools one should consider when determining the best approach to meeting a customer’s 
requirements, does not provide you with an insight as to whether these tools were used 
appropriately and achieved the goal of satisfying the customers. There are two mistakes 


here. 


First, establishing reverse auctions as a dashboard metric for the Chief of the 
Naval Supply Systems Command does not seem appropriate. This measure does not 
provide an accurate barometer as to whether the contracting function is operating 
“efficiently and effectively to deliver combat capability through logistics”. If used at all, 


reverse auctions should be a lower level metric. 


Second, using a reverse auction goal and measuring an office against that goal 
sends the wrong message. It may motivate contracting offices to fit a round hole into a 
square peg. Reverse auctions may be an excellent tool under the right circumstances. 
The key is to ensure that your contracting officers are trained and know the value of all 


the tools available to them. Performance Based Service Contracts (PBSCs), Small 
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Business, and Contract Closeout fall into this same category, with a major difference, 
ASN has mandated these metrics. ASN requires that 50% of all eligible service contracts 
must be performance based by FY 05. Again, similar to reverse auction, one can argue 
that a performance based service contract is just another tool for a contracting officer to 
consider when determining the best way to satisfy the customer. In addition, assigning a 
performance based services contract goal against a contracting office violates one of the 
tenets of selecting a metric; “the metric owner has the ability to influence the metric”. 
While contracting offices have a duty to train requirements generators in the benefits of 
performance-based contracts, they do not define the actual requirements. Many times 
the PCOs find themselves struggling to convert a non-performance based statement of 
work into a performance based statement of work while still satisfying the basic 
requirements of the service requested. There may be a strong argument to utilize a 
performance based contract, however, as GAO pointed out (GAO “Contract 
Management”: Guidance Needed for Using Performance Based Service Contracting), 
many still do not know all the steps required in defining a performance based service 
contract and more guidance is needed before determining the actual benefit of this type of 


contract. 


NAVSUP’s one metric that lends itself to efficiency is the Cycle Time Metric. 
However, this metric also has some problems. Cycle time measures the percentage of 
time you award a contract/order in less than 20 days for acquisitions less than or equal to 
$25,000 and 30 days for acquisitions between $25,000-$100,000. One of the problems 
associated with this metric is performance motivation. On many occasions, the customer 
does not require a contract/order within the 20 or 30-day goal (finding issues, required 
delivery dates, sending in a requirement 60 days before a contract is needed). On the 
other hand, some customers need their requirements filled immediately. PCOs must be 
capable of prioritizing workload without having to worry about meeting some arbitrary 
goal that does not provide a measure for efficiency and can seriously hinder your 
effectiveness. Most customers would not consider you too effective if you awarded their 
requirement in 19 days when they needed it in 2 days. This kind of metric may have a 
negative impact on customer relations and motivate poor business decisions. Not many 
managers would be happy to know that their PCOs would be willing to give up on the 
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idea of negotiating for a better price because they needed to get an award done within the 
cycle time goals. In addition, NAVSUP did not consider the transaction costs associated 
with gathering and managing this metric. The Standardized Procurement System (SPS) 
had to be modified and negotiators were required to track the reasons as to why they did 
not meet the cycle time goals and these had to be reported monthly. The only way to 
track the reasons is manually and is labor intensive. One has to question the value in 


gathering this metric when you analyze the cost of gathering it. 


All of NAVSUP’s metrics were established with no analysis of the management 
information systems available at each of their field offices. SPS does not have the 
capability to provide all of the metrics required. Also, all of NAVSUP’s metrics are 
“after the fact” or past metrics and does not look at Work-in-Process (WIP) or any future 
metrics. They utilize PMRS and MOCAS for some metrics and manual counts for others. 
One of the basic tenets of a good scorecard is to have a metric that looks as the past 
present and future in order to understand trends and motivate future performance. 

Unfortunately, some of the metrics established by FISC Norfolk does not look at 
efficiencies or effectiveness because some of the contracting offices do not have the 
ability to easily gather these metrics. The researcher is providing a model as part of his 
recommendations. The model proposed is assuming that all offices have the same 
management information system. This is a required element of the model because it will 


ensure that all have the capability of measuring. 


Some of the problems with FISC Norfolk’s metrics are the complexity of the 
measures. Many individuals the researcher interviewed understood the importance of 
gathering a specific measure, but did not understand how they were being graded against 
that metric. For instance, on page 11 of Appendix A, FISC Norfolk provides a sample of 
the efficiency index. It was difficult to understand where the figures for Data and Score 
came from. Most understood the fact that you would have different weighting assigned 
to different measures as some are more important than others, but the overall metric’s 
scoring sheet was sometimes difficult to understand. Also, some metrics fall into the 
same category as NAVSUP’s in that they measured processes that FISC Norfolk had 
little to no influence over. Specifically, the Portal/EMALL metric assumes that we have 


the ability to add contractors and catalogs to the EMALL. While it is true that we may 
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recommend and encourage contractors to have their catalogs added to the EMALL, we do 
not control the total process and many times it takes months in the DOD EMALL process 
chain to actually get a contractor/catalog added to the site. With regards to this metric, 
FISC Norfolk should concentrate only on the areas that they can influence (# of 
customers profiled) and disregard the measure of catalogs and Indefinite Delivery Type 


Contracts (IDTCs) added to the site. 


Another problem with the current metrics is the Customer Satisfaction measure. 
This measure is controlled by NAVSUP and requires the customers to access NAVSUP’s 
website and complete a customer satisfaction survey. Many customers have expressed 
dissatisfaction with this process mainly because they are being asked on a quarterly basis 
to fill out a satisfaction survey. In addition, there are many different customers 
associated with some Commands and it is often difficult to identify what specific 
customer within that Command needs to fill out the survey. Also, on some occasions, the 
sample size used to evaluate customer satisfaction was so small that the sample could not 
be relied on to provide an accurate measure of the effectiveness of customer support. 


d. What Type of Metrics Model is Required in Order to Measure the 
Efficiency and Effectiveness of a Navy Field Contracting Office? 


As stated in the answer to the primary research question, a balanced 
scorecard metrics model (Appendix B) is presented as a method for measuring the 
efficiency and effectiveness of a Filed Contracting Office. 


D. SUGGESTIONS FOR FURTHER RESEARCH 


While pursuing this thesis, the researcher found one major area that should be 
further researched: 


1. Measuring the Complexity of an Individual Contract Action 


Because measuring the productivity of a contracting office is difficult, the Navy 
has a tendency to lump together all of the different types of contract actions generated by 
a contracting office. As the research indicated, the Navy seems to think an “1102 is an 
1102”. The Navy also tends to think, “An action is an action”. The researcher knows 
from his experience that not all actions are the same and some are much more complex 


that others. While the researcher’s model segregates the current work-in-process by 
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action type, he does not measure the time it takes to complete a particular action. Further 
research into developing a time in motion study or some other study should be conducted. 
The ultimate measure of efficiency in a contracting office can only be attained when you 
have a full understanding of the type of work that is performed there. This type of 
analysis should always be done before comparing contracting offices against each other. 
The researcher recommends gathering a team consisting of an independent review 
official and Navy field contracting office managers and chartering them to establish a 
standard in which each contracting actions is measured. It should be noted that NAVSUP 
attempted to do this in 1994, but without an independent review authority. The team 
disbanded after the third meeting because they were all gaming the process to the benefit 
of their specific activity. By incorporating an independent review official, you may be 
able to subvert the political pressure that the prior team encountered. Once this is 
established, the Navy will have a better understanding of the productivity and efficiency 


of a field contracting office. 
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Corporate Metrics Alignment 


Corporate Metrics are intended to provide the Commanding Officer and the 
Acquisition Executive Board an overview of mission accomplishment and 
movement toward vision achievement 


while maintaining oversight of complex operations. Therefore, performance metrics 

should encourage performance improvement, effectiveness, efficiency, and appropriate 

levels of internal controls. Where practical, they should incorporate "best practices" 
related to the performance being measured and cost/tisk/benefit analysis, where appropriate. 
This section discusses the principles and concepts used in developing effective performance 
metrics for FISC Norfolk Acquisition Group operations. The underlying principles of metrics 
used within FISC Norfolk are based upon the writings of Mark Graham Brown is his books 
“Keeping Score” and “Winning Score”. Additionally, “The Balanced Scorecard- Translating Strategy into 
Auction” model of David Norton and Robert Kaplan will provide a recommended approach to 
ensure a complete set of metrics is promoted. This guide assumes reader familiarity with these 
topics, and as such will avoid detail review of their principles. 


T here continues to be increased emphasis on designing metrics to drive improvement 


Naval Supply Systems Command has promulgated specific guidelines for metrics in: 
e Customer satisfaction 
e Organizational performance 


e Workforce excellence 


Additionally, the Acquisition Group Business Plan identifies critical initiatives. The key elements 
of the performance metrics package to meet these initiatives should address the following: 


e Attaining Organizational Alignment 


e Improvements in quality, cost reduction and/or avoidance 
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e Meeting customer requirements 
e Meeting NAVSUP requirements 


e Identification and development of resource requirements 


The Process 


The first step in developing performance metrics is to involve the people who are responsible for 
the work to be measured because they are the most knowledgeable about the work. Within the 
Acquisition Group, these people are identified as the initiative owners within the Acquisition 
Group Business Plan. Once identified and involved, they: 


e Identify critical work processes and customer requirements. 
e Identify critical results desired and align them to customer requirements. 
e Develop measurements for the critical work processes or critical results. 


e Establish performance goals, standards, or benchmarks. 


The establishment of performance goals can best be specified when they are defined within three 
primary levels: 


e Strategic Objectives: Broad, general areas of review. These generally reflect the highest 
goals based upon mission accomplishment and vision achievement. These metrics will 
comprise the corporate level metrics. 


e Operational Criteria: Specific areas of accomplishment that satisfy major divisions of 
responsibility within a function. These metrics are generally mission focused and form 
the underpinning of strategic metrics. 


e Warning Measures: Metrics designed to drive improvement and characterize progress or 


problems within each operation. These are specific quantifiable goals based on individual 
expected work outputs of critical processes or contributory functions. 


The SMART test is frequently used to provide a quick reference to determine the quality of a 
particular performance metric: 


S = Specific: clear and focused to avoid misinterpretation. Should include measure assumptions 
and definitions and be easily interpreted. 
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M = Measurable: can be quantified and compared to other data. It should allow for meaningful 
statistical analysis. Avoid "yes/no" measures except in limited cases, such as start-up or systems- 
in-place situations. 

A = Attainable: achievable, reasonable, and credible under conditions expected. 


R = Realistic: fits into the organization's constraints and is cost-effective. 


T= Timely: doable within the time frame given. 


Types of Metrics 
Quality performance metrics allow for the collection of meaningful data for trending and analysis 
of rate-of-change over time. 


Examples are: 


e Trending against known standards: the standards may come from either internal or 
external sources and may include benchmarks. 


e Trending with standards to be established: usually this type of metric is used in 
conjunction with establishing a baseline. 


e Milestones achieved. 


"Yes/no" metrics ate used in certain situations, usually involving establishing trends, baselines, or 
targets, or in start-up cases. Because there is no valid calibration of the level of performance for 
this type of measure, they should be used sparingly. 
Checklist for Evaluating Corporate Metrics 
The “scorecard” concept will be used by FISC Norfolk to summarize the results of critical 
metrics. The following checklist from “Waning Score” will assist in evaluating the scorecard: 

e A reasonable number of metrics is less than 20 

e Measures are balanced against past, present and future time perspectives 

e All key success factors and business initiatives have at least one metric 

e Weighting will be used to highlight critical metrics 

e Metrics should avoid being manipulated 


e Indices should be used to ease data review and aid in analysis 
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Measutes, and initiatives are aligned with the priorities of customers, employees and 
stakeholders 


Process measures cottelate to outcome measures 


Business fundamentals will be included in the scorecard 
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Scorecatd Metrics 


The Scorecard is designed to provide a summary view of the underlying 
corporate metrics to Commanding Officer and the Acquisition Executive 
Board. 


departments have metrics and data charts that should be maintained at the departmental 
level, many do not tise to an ESC command level review. To pare-down these mettics, a 
scorecard has been designed for the command level. Whenever performance as 
indicated on the scorecard warrants more detailed review, the underlying metrics and graphics will 


be used 


T here is an effort to streamline the FISC Norfolk metrics brief. Although many 


This section discusses the scorecard and its associated metrics for FISC Norfolk Acquisition 
Group operations. .It will guide the Group in data collection and presentation to ensure timely, 
consistent data collection and presentation 


The Scorecard 

The first step in developing the scorecard is to determine the performance mettics to be 
displayed. Within the Acquisition Group, these metrics are those provided by the initiative owners 
and approved by the Acquisition Executive Board. 


The initiatives are aligned with the Naval Supply Systems Command Strategic Plan Goals. The 
strategic metric, objectives, and timing associated with each of these initiatives will form the basis 
for the Scorecard metric. 


The Scorecard will utilize a “Red/Yellow/Green” format to summarize performance. 
Determination of the level of performance and allowable variation will set the criteria for 
appropriate color coding, while trending observations will be require two consecutive movements 
in the same ditection. 


Key Success Factors 

The Key Success Factor mettics are indices reflecting both mission accomplishment and forward 
movement on vision attainment. The standards for growth metrics are typically determined by 
the fiscal month of the year, e.g. in December we should have accomplished 25% of the goal; or 
by the initiatives plan for implementation. Minimum grades or Statistical Process Control sets the 
standards for process metrics as appropriate. Individual metrics ate not coded as 
“Red/Yellow/Grteen’. Instead, the indices are rated, with the standard set by the summarty of the 
individual metric within the respective index. “Green” will be assigned to any index achieving or 
within five percentage points of that standard, “yellow” will be assigned to any index scoring 
between fifteen 
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and five percentage points, and “red” will be assigned to all others. All initiative owners will 
report their metrics to ABM to support monthly reporting. All sites will provide their specific 
data, such as % of training hours or % mettics, to initiative owners at least two days prior to the 
ABM due date for command submission. 














e Effectiveness Index: This consists of three weighted individual metrics. 

Mettics Initiative Owner Frequency Standard Source 

# of Customers Marketing Monthly A growth metric determined by the annual | Manual count 
Profiled plan. 

Catalogues and Portal Monthly A growth metric determined by the annual | Manual count 
IDTCs on Portal plan 

Customer CRM Monthly A ptocess metric. The standard is always NAVSUP Database 











Satisfaction Index 95% Highly Satisfactory or above. 


° Efficiency Index: This consists of four weighted individual metrics. 





Metrics Initiative Owner Frequency Standard Source 
Intranet Hits Intranet PM Monthly A growth metric determined by the Website Hit Counter 
desired goal of each employee accessing 


the site every other day. 








Customers on Portal Portal Monthly A growth metric determined by the annual | Manual count 
plan 
e-Transactions Index | Innovative, Monthly A process metric. ‘The standard is always OSD Paperless Metrics Report 
Streamlined 92%. The metric will use the OSD report, | from ABM 
Acquisition Processes but only include FISC-controlled 
and Tools processes of solicitations, award and 


closeouts. In final calculations, all 
subsidiary metrics associated with the 
awards metric will be counted individually. 








“% Paperless Review COC of each site Monthly A growth metric determined by the Manual count from the Review 
Board desired goal of achieving 95% of review Board log. 
boards held by electronic media. 



































° Sustainment Index: This consists of four weighted individual metrics. 
Metrics Initiative Owner Frequency Standard Source 
Series Standards Personnel Monthly A growth metric determined by the Manual 
Profiled Development desired goal of profiling ten critical 
Series/specialties. 
% IDPs Executed Personnel Monthly A growth metric determined by number of | Manual count 
Development people in the ten critical 
series/subspecialties 
Training Personnel Monthly A growth metric determined by Manual count from Access 
requirements Development identification of resources identified for Training Database 
resourced the ten critical series/subspecialties 
Training Hours Met Each Detachment Monthly A growth metric determined by the HRO 
desired goal of achieving 100% of the 40- 
hour requirement. 














Initiative Metrics 

Each initiative will have several initiatives associated with them. The scorecard will capture at least 
one metric for them The standards for growth mettics are typically determined by the fiscal 
month of the year, e.g. in December we should have accomplished 25% of the goal; or by the 
initiatives plan for implementation. Minimum grades or Statistical Process Control sets the 
standards for process metrics as appropriate. “Red/Yellow/Green’” indicators are assigned to 
individual metrics. 
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Monthly metrics will be reported for the Acquisition Group as a whole. Individual sites will 
report quarterly metrics, with the: 


e First month of each quarter reflecting Philadelphia, 


e Second month reflecting Norfolk, and 


e Third month reflecting Washington metrics. 






































Metrics Initiative Owner | Frequency | Standard Source 
Customer CRM Quarterly A process metric with standards set by NAVSUP. Green equals attainment | NAVSUP 
Satisfaction of NAVSUP goal of highly satisfactory rating. Yellow equals a grade within | Database 
Survey five percentage points of that standard, currently set at 90% 
Portal Product Monthly A growth metric determined by dollar value of sales on the portal divided by | IBM for Portal 
Customer Development/ the total sales of those same customers. Green will equate to within 90% of | Sales, Dynamic 
Market Share Portal plan, yellow to within 80% of plan. The plan will be established as the Reporting for 
Development projected annual sales of all portal customers, to suppliers that are on the Total Sales 
and Execution portal.. Annual sales will be projected based upon the starting month of the 
customer portal usage. 
% Innovative, Quarterly A growth metric with the baseline set for each site based upon previous FY | PMRS reports 
Performanced | Streamlined results. Green will be reported if the score exceeds 5% above the prior year, | from ABM 
Based Acquisition yellow if it is within 5% plus or minus of the prior year. where B1D= 
Contracts Processes Yes/Total 
Contracts 
Awarded 
Reverse Innovative, Monthly A process metric determined by the number of planned auctions compared Manual 
Auctions Streamlined to the NAVSUP goal. Green equals planned attainment of the goal. Yellow 
Acquisition equals planned auctions within 20% of the goal. 
Processes 
Self- Self-Assessment | Quarterly A compilation process metric combining PMR, SAP, Large Procurement FISC Norfolk 
Assessment Program and Special Interest Item QA plan scores. Any red score or two yellow Det 
Findings Management scores for any of these categories will equal a red for the metric. Any yellow | Philadelphia 
score in any category will result in an overall yellow. Green will be assigned | QAPM 
if all areas score green. Database 
Within PMR, total audits within 90% of schedule will be green, while within 
85% will be yellow. 
Within SAP, excess error rates in one area will lead to a yellow rating, while 
two will lead to a red 
Within Large Procurement, excess error rates in two areas will lead to a 
yellow, while three will lead to a red. 
Within Special Interest, error rates greater than 10% will equal a yellow, and 
greater than 20% will equal a red 
Small Small Business Quarterly A process metric with standards set for each category. Failure to achieve PMRS 
Business Managers two categories will equate to a yellow, while failure in three will equate to a 
red 
Overage Contract Monthly A process metric with green achieved when monthly closeouts equal or DRID 53 and 
Contract Closeout exceed a standard set at the total overage on hand divided by 12. Yellow Workload 
Closeout will be assigned if closeouts are within 10% of that standard. All sites must Staffing 
reflect green for an overall grade of green. Any site with a ted will result in Reports 
a grade of red. 
SPS Upgrade Procurement Quarterly TBD TBD 
Effectiveness | Systems 
Management 
% of Portal Monthly A process metric determined by the number of transactions on the portal IBM for Portal 
Customer Development divided by the total number of customer Purchase Card transactions. The Actions, 
Requirements | and Execution standard will be set: as the percentage of transactions expected from their Dynamic 
met via the catalogue and contract vendors on the portal as determined by the % those Reporting for 
Portal suppliers achieved in the past fiscal year. A score of 90% will be green, Total Actions 
while less than 80% will be red. 
% of FISC Market Quarterly A growth metric determined after critical assessment of potential for Manual 
Markets Managers requirements consolidation through contracts. The annual plan to analyze 
Sourced and award contracts by FSC will set the standard 


93 








NMCI Order 
Fulfillment 
Rates 


Employee 
Satisfaction 


NMCI 


HRSC 


Quarterly 


Annual 


A process metric determined by comparing actual orders executed to the 
program manager’s plan. Green will indicate the orders executed are 
running to plan. Yellow will indicate 0%-10% behind plan. 

A process metric provided through command assessment 


Manual 





HRSC Survey 





Alignment 


Metrics 
Alignment 


Quarterly 


A process metric reflecting results of the quarterly alignment survey. 

Standards are set for absolute and relative scores: 

(1) — Absolute: If the lowest score is below 35 it will be yellow, if below 25 

it will be red 

(2) Relative: If the difference between the highest and lowest score 

exceeds five, this will be yellow; if it exceeds ten, this will be red 

If both parameters ate either red, or if there is one yellow and one red, the 
alignment metric is ted. If only one parameter is red, the metric is 
yellow. If only parameter is yellow, the metric is green. 


Alignment 
Survey 
Database 








Intern 
Vacancies 





ABM 


Monthly 





A growth metric determined by the number of intern billets without fills or 
selections. Vacancies in excess of 10%-20% of authorization will be yellow, 
greater than 20% will be red. However, after April of each fiscal year, 
yellow will be 0%-10% and red will be greater than 10%. 





Manual 
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Reporting Requirements 


The Scorecard requires timely reporting of accurate data in order to be a 
useful tool. 


must contribute their data in a timely and accurate manner. The ABM division is 

responsible for the compilation of data for further submission to the FISC Norfolk 

metrics manager. These metrics are usually due near the end of the first full week of each 
month, although a specific schedule is promulgated with exact due dates. It is critical that each 
mettic owner become familiar with the date reporting format and schedule to ensure the 
Acquisition Group Scorecard is as accurate as possible for the monthly review and subsequent 
posting to the Intranet. 


T he monthly Scorecard’s development is a team effort. Acquisition Group metric owners 


Reporting Format 

The following worksheet will be used to provide standardized input to the ABM. Owners may 
submit their input via e-mail; however, they should ensure the data is reported via the format of 
this section. 


Key Success Factors 
1. Effectiveness Index:: This consists of three weighted individual metrics. Each site will 
provide input from their offices on customer profiling and new catalogues to the 
Acquisition Group initiative lead. The CRM lead will obtain data from the NAVSUP 
database on the customer satisfaction index for all sites. It will consist of the “% of all data 
scores received for all elements that are rated highly successful or better. 





Month Site 








Comments 








Metrics Weighting Plan Actual 





Data | Score | Weight | Data | Score | Weight 





a. # of Customers 30 
Profiled: 





b. Catalogues and IDTCs | .30 
on Portal: 





c. Customer Satisfaction 40 
Index: 








Index Total Total 





Color Trend 
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2. Efficiency Index:: This consists of four weighted individual metrics. Detachment Philadelphia 
will provide intranet data directly to ABM. Each site will provide input from their offices on 
customer portal to the initiative lead. Each site will forward their e-transactions data from the 
OSD Paperless Metrics Report to ABM. The ABM will calculate the final number based upon 
the total of the actions in the following categories: 


© Solicitations issued 

e Awatds and modifications issued to: 
a. Files 
b. Originator 
c. ACO 
d. DFAS for payment 
e. DFAS for accounting 
f. Contractor 

© Contract Closeouts 

Each site will review their CRB log to provide the data for the number of paperless and total CRBs. 





Month Site 








Comments 


Metrics Weighting Plan Actual 
Data | Score | Weight | Data | Score | Weight 








a. # of Intranet Hits 25 
b. Customers on Portal 25 

d. % Paperless Review 10 75% | 80 8 
Boards 


Index Total Total 
Color Trend 





























3. Sustainment Index: ‘This consists of four weighted individual metrics. The Personnel 
Recruitment, Development and Retention lead will identify how many series have been 
profiled. Each site will report how many IDPs are due/have been reviewed and executed 
based upon profiled series standards. They will also report how many of the training 
requirements identified in the completed IDPs have been sourced and resources identified. 
Fach site will provide input from their offices to ABM on the % of personnel requiring and 
completing the 40-hour training requirement. 













Comments 








Metrics Weighting Plan Actual 





Score | Weight 





a. Series Standards Profiled 


b. % IDPS executed 
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c. Training Requirements 


Resourced 





ET 


Index 


.10 


Total 


Total 





Color 


Trend 














Initiative Metrics 
4, Customer Satisfaction Survey: 





Comments 








Quarter 


(Q/FY) 
1: 


3: 








Highly Sat. | Superior 












































Color: Trend: 
5. Portal Customer Market Share: 
Month Site 
Comments 
Customers Months Annual Plan Actual YTD Performance 
on Portal Portal Sales | Portal Sales Plan YTD Portal Sales 
Est. 
FISC Norfolk 579,026 
NNSY 249,396 
SIMA 79,792 
91,044 
FOSSAC 
NAVTRANS 
DET WASH 25,000 
DET PHIL 3,000 
NAVICP 100,000 
NAVSUP ey ee Se (ee) 
NDW 25,000 
Total 1,152,258 0 0 
Percentage | !Zero Divide | Color Trend 
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6. % Performanced Based Contracts 


— 








Comments 
Philadelphia Norfolk Washington 
Plan 
a YOY 
i _ 
Actual 


a. Performanced Based 


Contracts 








b. Total Contracts 
c. Percentage 





Difference to Plan 





Color 





Trend 








7. Reverse Auctions: 











Month Site 

Comments 

Goal 10 4 

Item Site Date Status Estimated $/Savings 


(Planned/Complete) $ 





























8. Self Assessment Findings 


























































































































a. PMR 
Month Site Norfolk & Charleston 
Comments 
Actions Scheduled Completed % 
Desk Audits 
PMR and Site Audits 
Overall Color Trend 

b. QA Plan 
Quarter Site | 
Comments 
SP1 SP2 SP3 SP4 SP5 Color SI1 Color 
Lc LC2 LC3 LC4 LC5 LC6 LC7 Color 
Overall Color Trend 

9. Small Business: 
Quarter Site 
Comments 
SB SDB WOSB HZSB Vet HBC Month 

Goal 34.4 2.5 5.0 2:5 3.0 : 
Actual 
Overall Color Trend 
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10. Overage Contract Closeout 


= oe 


Comments 





Philadelphia Norfolk Washington NNSY 





On Hand 





On Hand/12 





Overage 


— | 
=| 
—— 
— 


Overall Color Trend 














11. SPS Uperade Effectiveness Quarterly TBD TBD 


12. % of Customer Requirements met via the Portal 


— ia 


Comments 





Customers Months | Annual Plan Actual YTD Perfomance 





on PC % of | Plan YTD | Portal % 
Portal Actions Est. Actions 
FISC Norfolk 











NNSY 
SIMA 
PWC 








DET PHIL 





NAVICP 





FOSSAC 
NAVTRANS 
DET WASH 


NAVSUP 
NDW 








Total 0 0 














Percentage Color Trend 
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13. % of FISC Markets Soutced 


Comments 





Market 


# of FSCs 














14. NMCI Order Fulfillment Rates 











Quatter 


Comments 


Increment 


Site 


Deobligation Mods 





1A) 


Planned 


Planned 





Actual 
Planned 
Actual 





Color 





Planned 
Actual 








Trend 
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15. Employee Satisfaction 























16. Alignment 
Quarter Site | 
Comments 
Strategy People Customer Process 
Score 
Lowest Score Color 
Biggest Color 
Difference 
Color Trend 














17. Intern Vacancies 


Month 





Comments 












Billets Vacancies 
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APPENDIX B. METRICS 








an 
FOR MEAS 3 


BALANCED SCORE 
STRATEGIC METRICS 
OPERATIONAL METRICS 
CUSTOMER METRICS 


EMPLOYEE METRICS 
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STRATEGIC METRICS 











COMPARISION OF 2 
ACTIONS OVER 2: 








BFY01 Actions 



































@ FY02 Actions 

















Source: PMRS Database DF62N 
Does not include actions where total dollars are “0” 
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COMPARISION OF 
DOLLARS OVER 





$0 Jan 


mFY01 OBS $313 
mFY02 OBS $290 





















































Source: PMRS Database DF62N 


LARGE PU 
TOP 10 SUPPLIER. 


TOTAL 
SUPPLIER FSC DESCRIPTION DOLLARS 


Electronic Data Systems Corp Other ADP & Telecommunication Svcs $60,422,537 
Dell Marketing LP ADP Central Processing Unit $44,871,014 
Innovative Logistics Technique Logistics Support Services $33,477,534 
General Dynamics Gov Sy RDTE/Other Defense Adv Tech Dev $32,540,992 
AMSEC LLC Maint & Repair of Eq/Miscellaneous Equip $25,005,445 
Goodwill Industries of SE WI Other ADP & Telecom Svcs $23,520,814 
Booz Allen & Hamilton, Inc. Tuition, Registration & Membership Fees $21,854,247 
Mantech Systems Engineering Co. | Other ADP & Telecommunications Svcs $20,964,620 




















San Diego City College Training/Curriculum Development $19,485,168 





Lockheed Martin Services In.c Training/Curriculum Development $18,755,870 
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Actions Obligations 





Supplies 163 $59,858,299 





Services 1424| $650,497,165 














LARGE PURC: 
TOP 10 FSC’S BY 





Other ADP & Telecommunication Services (D399) 





Logistic Support Services (R706) 





Tech Rep Svcs/Aircraft Gnd Handling Equip (L017) 





Other Medical Services (Q999) 





Maint & Repair of Eq/Misc Equipment (J099) 
Engineering Technical Services (R425) 





RDTE/Other Defense-Adv Tech Dev (AD93) 





Training/Curriculum Development (U008) 
Maint & Repair of Eq/Ships-Sml Craft-Docks (J019) 








Tech Rep Svcs/Aircraft Comps. & Accys (L016) 
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A monthly update of the actual competition rate versus the goal 


% 
100 





80 


60 


40 


20 
































0 





ma MONTHLY 
CUMULATIVE 
— OBS GOAL 

































































Goal 82% 


E-BUSI 
(Paperless 


























WELEC PRs 
@ELEC SOL 
ELEC AWDS 
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Description 


ANTICIPATE, 
OBLIGA 


Customer 


FY03 


Obligations 


FY04 


FY05 


FY06 


FY07 





ETS for C4l 


BPR/ABC 
SUPPORT SVCS 


SPAWAR $86M 


JFCOM $25M 


$15M 


$ 16M 


$17M 


$18M 


$17M 





ETS 


NAVSHIPSO $160M 





CA Food, Mat Magt, 
Custodial 


BUMED CAL REG 
$50M 





GTMO Telecom. 


NTCSS Fit Support 
Svcs 


Curriculum 
Development 


NCTAMSLANT 
$29.9M 


SPAWARSYSCEN 
$154M 


NETPDTC $29M 


$30.7M 


$32.0M 





Navy College 
Learning Prog. 


NETPDTC $49M 





Integrated Supply 
Support 


PNS Great Lakes 
$25M 

















CONTRACT 
STA 

















MAR 


APR 





1293 


1858 























1200 








1400 























* Includes 1,412 SAP actions 
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F/Month Year-To-Date 





582 

0 
177,179 
0 
16,010 
24,509 
0 

0 
168,898 
258,720 
25,683 
671,581 












































Data FISC NAVSUP ASN 
Category (GTE $25K) | (GTE $100k) Website 


Beg Balance 
Eligible 


New Eligible this 
Period 





4129 1561 3269 








Contracts Closed 
this Period 


Ending Balance 
Eligible 


Overage Ending 
Balance 


336 96 96 





3913 





1569 








* 205 number includes contracts imported for months of March, April and May 2002 
(3 months added to ASN website on 11 June 2002) 
** ASN website still to be updated to reflect completed closeouts 
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GOALS/ACHIE\ 





FISC Norfolk 
$ $ 


Det Phila. 


FISC 
Norfolk 
Target 


FISC 
Norfolk 
Actuals 


Det 
Phila. 
Actuals 





Total to U.S. Firms 


$953,909,039 


$514,736,321 





Small Business 


$342,156,226 


$133,504,458 


34.4% | 35.9% | 25.9% 





SDV 


$580,793 


$245,610 


3.0% | 0.1% | 0.05% 





Small Disadvantaged 
Business 


$140,693,354 


$77,374,168 


11.2% | 14.8% | 15.0% 





Women-Owned Small 
Business 


$63,282,690 


$45,871,953 


3.8% | 6.6% 8.9% 





HUB Zone 


$15,265,341 


$1,718,730 


2.5% | 1.6% 0.3% 





HBCU/MI 








$18,462,301 


$18,462,301 











22.5% | 77.1% | 77.8% 





BUSINESS DEVEL 





Customer 


Description of Initiative 


Status 





OSD/C31 PIMS 
International 
Outreach 


Partnership for Peace 
Information Mgmt 
Systems Svcs Support 


Contractual planning mtg to be 
scheduled for support of PIMS 
program. 





JFCOM (J8) 


Technical Support Svcs 


Met 5/6; expect SOW NLT 6/30. 





CNRNE 


Husbanding Contracts 


RFP issued 6/5/02. 





Naval War College 


War Gaming Support 
Svcs 


BAA drafted. Pre-Solicitation 
Conference 6/25/02. 





Joint Forces 
Intelligence 
Command 








General Contracting 
Support 





J4 advised that JFIC will be 
forwarding some requirements to 
us. 
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Inactive Ships 

NEMMI 

CNET 

JFCOM 

FMS 

Travel Management 
Shipboard Copier 

A-76 

Acquisition Reform Office 
NAVICP FMS Logistic Support 
Shop Towels 

Task Force Excel 








OPERATIONAL 
METRICS 
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Awards 47 | 49 48 | 53 | 89 | 67 
Goal Achieved 25 | 30 45 | 50 | 66 | 61 
-~ Monthly % 64% | 63% | 53% | 61% | 90% | 94% | 94% | 80% | 91% 

































































TIMELY SERVICE — LT $25K — 20 DAYS OR LESS 






































May 
= Awards 15 | 12 | 18 | 14 | 24 | 24 
Goal Achieved 10 | 12 | 16 | 13 | 22 | 22 
=> Monthly % 83% | 71% |100% 67% |100%| 89% | 93% | 92% | 92% 




































































TIMELY SERVICE — $25K — $100K (SAP) - 30 DAYS OR LESS 
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COMPETITIVE S 


A-76 Metrics * 





Cost 
Comparison | In Process | Completed 
Method Used 


Full Study 23 


Gov't 
Retained 





65 and Under 31 





10 and Under 47 





Direct 16 
Conversion 














* Plus 33 Cancellations of Studies 


A-76 Metrics * 


Studies Average 


Process Completed PALT 


PALT 
Range 





Full Study 23 11.74months 


6-13 months 





65 and Under 31 3.45 months 


1-6 months 





10 and Under 47 3.04 months 
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1-6 months 





WORK-IN-PROCES 


NEW TYPE | Dos 
PROCUREMENT 33 
90 


TYPE II Dos 
41 


TOTAL FOLDERS - 281 


WORK-IN-PROCES 
ACQUISITION STi 


NOT ELIGIBLE 
8(a) SET-ASIDE FOR 
7 COMPETITION 
3 


SOLE SOURCE 
103 
FULL & OPEN 
COMPETITION 
146 


SMALL 
BUSINESS 
22 


TOTAL FOLDERS - 281 
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TOTAL FOLDERS - 281 


NEW PROCUREMENT; 





Sole 
Sole | Competitive | Source 
Measurement _|Competitive | Source Cl CI 
PR rec’t date 
thru sol. Issue 30 16 4 1 

date 

From sol. issued 
date actual 
closing date 
After sol. closes 
& TE is 
requested 





Unsolicited 
Phase 





Solicitation 
Phase 





Tech Eval 
Phase 





Negotiation 


Phase After sol. closes 





Total 
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COMPETITIVE (29) 
Red 


" Stratification 
Yellow 
@ 0-30 days 
O 31-45 days 
@ Over 45 days| 

















COMPETITIVE Cl (5 


Yellow 
1 








Green Stratification 
3 

@ 0-15 days 
© 16-30 days 


@ Over 30 days 











COMPETITIVE (6) 


Stratification 

@ 0-30 days 

© 31-45 days 
@ Over 45 days| 








4 





COMPETITIVE Cl (3) SOLE SOURCE Cl (0) 








Stratification 
@ 0-15 days 
© 16-30 days 
@ Over 30 days 








Yellow 
1 
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COMPETITIVE (7 


Yellow 
1 


TECH EVAL f 
(8 PROCURE 


SOLE SOURCE (0 











COMPETITIVE Cl (1 


COMPETITIVE (4 





Stratification 
@ 0-30 days 


© 31-45 days 
SOLE SOURCE Cl (0 


@ Over 45 days| 











SOLE SOURCE (12 


Yellow 
1 


TECH EVAL 





Stratification 
@ 0-30 days 
O 31-45 days 
@ Over 45 days 














COMPETITIVE (2 





WITHOUT SOLE SOURCE (0 


TECH EVAL 





@ 0-15 days 
O 16-30 days 
@ Over 30 days 
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DELIVERY 


TYPE | DOs (19) TYPE II DOs (26) 
Red NON-COMPETITIVE 


5 


Gieea Green 
14 26 


@ 0-15 days @ Over 15 days © 0-25 days © 46-45 days © over 45 days 


TYPE | DOs (14) TYPE II DOs (15) 





Yellow 


1 COMPETITIVE 


Red 


4 Yellow 


2 


@ 0-25days O 26-45days @ Over 45 days @ 0-25days O 26-45 days @ Over 45 days 
31 


OPTIONS (12) 


Yellow 
1 


@ 0-30days O 31-60days @ Over 60 days @ 0-15days @ Over 15 days 


CHANGE ORDERS (43) SAP_(38) 


Yellow 
8 





Yellow 
Green - 
2 Red 
11 


@ 0-30days © 31-60days ® Over 60 days @ 0-30days © 31-60days @ Over 60 days 
32 
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Simplified Acquisit 
Assessment Fin 
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Philadelphia 


All Team Leaders are responsible for signing all SAP Actions 


Weekly training sessions on SAP will be conducted starting Oct 01 
for 1105s and Team Leaders 


Modify the GSA Template Memo 
Establish SAP Toolkit and post on Intranet 


Team Leaders and negotiators must ensure all documentation and 
chronology of events are in file 





CUSTOMER METRICS 
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Total 
Customer/UIC Actions 


NAVAL AIR SYSTEMS COMMAND HEADQUARTERS (N00019) 
NAVAL EDUCATION & TRNG PROF MGMT SUP ACT (N68322) 
NAVAL SHIPYARD, NORFOLK (N00181) 

SPAWARS (N66001) 

FLEET TECHNICAL SUPPORT CENTER ATLANTIC (N65912) 

















NAVAL SEA LOGISTICS CENTER CODE 134 (N65538) 

JOINT WARFIGHTING EXP BATTLE LAB JWEBL (N3188B) 

NAVICP (N00391) 

JTASC (N39792) 

NAVAL AIR TECH DATA & ENGR SVC COMMAND, NAS NORTH ISLAND 














JAN 02 Customers Provided. PMRS generates inaccurate FY02 data due to DD350 block changes (from Section G to Block E4). 
Working on solution. 





Customer/UIC Total Dollars 
NAVSUPSYSCOM HQ (N00023) $51,909,654 
NAVAL INVENTORY CONTROL POINT (N00391) $50,168,532 
NAVAL SHIPYARD, NORFOLK (N00181) $44,906,846 
JOINT WARFIGHTING EXP BATTLE LAB JWEBL(N3188B) $36,439,804 
COMMANDER IN CHIEF PACIFIC FLEET, NAVBASE (N00070) $34,428,259 
NAVAL EDUCATION & TRNG PROF MGMT SUP ACT (N68322) $31,291,146 
NAVAIRSYSCOM HQ (N00019) $25,798,773 
SPAWAR CHESAPEAKE (N68561) $24,466,863 
SERVICE SCHOOL COMMAND, NTC (N0580A) $21,712,997 
SPAWAR CENTER (N66001) $20,448,150 


























JAN 02 Customers Provided. PMRS generates inaccurate FY02 data due to DD350 block changes (from Section G to Block E4). 
Working on solution. 
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WHAT DO WE BUY FOR Ol 


CUSTOMER 


# Actions 





NAVAIR — N00019 


NATPDTC — N68322 
































WHAT DO WE BUY FOR O 





CUSTOMER 


NAVAL SHIPYARD 
NORFOLK — N00181 


# ACTIONS 











FLEET TECHNICAL 
SUPPORT CENTER 
ATLANTIC — N65912 
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WHAT DO WE BUY FOR C 
CUSTOMER 


CUSTOMER # ACTIONS 


NAVSEA LOGISTICS CENTER 
CODE 134 — N65538 











SPAWARS — N66001 




















WHAT DO WE BUY FOR O 
CUSTOMER 





# ACTIONS 


JWEBL N3188B 














NAVAL Al TECH DATA & 
ENER SERVICE COMMAND — 
N32379 














JTASC — N39792 
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WHAT DO WE BUY FOR C 
CUSTOMERS 


CUSTOMER # ACTIONS 
NAVICP — N00391 























EMPLOYEE METRICS 
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Series Breakdown 











m9/30/99 
WM9/30/00 
m9/30/01 
[Current 
































TRAINING STA: 

















0 hrs 

@ 1-8 hrs 

@ 9-24 hrs 
25-39 hrs 
40+ hrs 
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16,000 7 


14,000/" 
12,000+"" 
10,000+"" 





Budget 


1102/1105 TRAINII 





Obligated 


Expended 








15,000 





11,000 





7,800 





Source:OTF Report 





16,000 


14,000+" 
12,000+"" 


10,0004" 


8,000+" 


6,000" 
4,000; 
2,000+" 

0 





Budget 


Obligated 


Expended 








7,247 





6,955 





403 





Source:OTF Report 
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